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There was a need for this project because: 

■ with a few exceptions, thinking skills are not 
explicitly built into education and training for 
post-16 learners in England 

■ there is no established basis forchoosing 
between thinking skill frameworks forgeneral 
orspecific applications 

■ little is known, although much is claimed, 
about the potential of thinking skills approaches 
for helping to raise the quality of education 

and training delivery for post-16 learners. 

We decided to: 

■ considerall kinds of thinking and learning which are 
to some degree under conscious control 

■ exclude unsystematic ways of classifying thinking skills 

■ include frameworks designed for different purposes 
and from different perspectives - broadly educational, 
psychological and philosophical 

■ apply a consistent set of evaluation criteria, focusing 
on underlying values, theoretical and practical aspects 
and communicability 

■ summarise and evaluate 35 thinking skill frameworks, 
nine of them in more detail than the rest 

■ explain our understanding and use of key terms, 
while recognising differences of opinion and trying to 
distinguish substantive from less substantive issues. 

We recognised that: 

■ research reviews about what makes thinking skills 
approaches effective highlight (a) learner engagement, 
(b) thinking about thinking, and (c) the strategic 
management of thinking 

■ if teachers are to engage with and use thinking skill 
frameworks, these must be clearly expressed and 
situationa lly appropriate, and should conveya degree 
of both affirmation and challenge 

■ if learners are to benefit from thinking skills 
approaches, they need to develop a deeper 
understanding of learning and instruction and 
appreciate the value of thinking skills in daily life. 



We found that: 

■ we could classify frameworks of thinking skills under 
four headings: 

■ all-embracing frameworks covering personality, 
thought and learning 

■ instructional design frameworks 

■ frameworks for understanding critical and 
productive thinking 

■ explanatory models of cognitive structure and/or 
cognitive development 

■ the largest of the fourgroups is the instructional 
design family, where the strong and continuing 
influence of Bloom’s taxonomyof educational objectives 
(1956) is apparent. Anderson and Krathwohl’s (2001) 
revision of Bloom’s taxonomy has much to commend it 
(see Section 3.3.2) 

■ many critical and productive thinking frameworks 
include process categories which resemble Bloom’s 
analysis, synthesis and evaluation; there is also 
general agreement about the need to manage thinking 
strategically, aided by appropriate dispositions 
(habits of mind) 

■ Halpern’s 1 work on critical thinking skills and 
dispositions (see Section 3.4.2) stands out as being 
coherent, broad in scope and accessible forteachers 
and learners 

■ although theorists have different ideas about the 
structure and development of cognitive abilities, they 
agree that these are multidimensional and modifiable 

■ Pintrich’s (2000) framework for self-regulated 
learning (Section 3.5.2) represents an integrated 
dynamic conception of how thinking and learning 
can be developed 

■ in recent years, a number of all-embracing frameworks 
have been constructed, most notablythe work 

of M a rza no (2001a, 2001b; see Section 3.2.2). 

Because: 

■ thinking skill frameworks can be used for many different 
purposes; and 

■ thinking skill frameworks are constructed on the basis 
of at least 15 different classificatory principles 

we recommend different frameworks for different 
applications: 

■ academic study 

■ instructional design 

■ developing pedagogical theory and practice 

■ consultancy 

■ assessment 

■ research and evaluation. 



i 

Where a date is not appended to an author's name, the reference is to his 
orherwork overall. 





We have developed a new integrated framework 
to bring out how: 

strategic and reflective thinking can interact with 
and enhance the quality of information gathering, 
building understanding and productive thinking 
(see Section 4.3). 

Our integrated framework has these advantages: 

■ it is expressed in clear, simple English 

■ it is consistent with, but extends the range of, Bloom’s 
taxonomy (1956) 

■ it is consistent with frameworks of critical thinking, 
and with models which include 'habits of mind' and 
self-regulation 

■ it is consistent with conceptions of creative and caring 
thinking (including emotional intelligence) 

■ it is consistent with, although more simply structured 
than, Marzano’s new taxonomy (2001a, 2001b) 

■ it is consistent with, but extends the range of, 
the English National Curriculum categories 

■ it is consistent with the ways in which a sample 

of college teachers think of subgroups of thinking skills 
as differing from each other 

■ it is easily understood and remembered, making it 

a suitable mental model for practical use by teachers 
and learners 

■ it can be consistently applied and is an adaptable 
multi-purpose tool. 

When we applied our framework to key skills 
knowledge objectives, we found that: 

■ building understanding and productive thinking are 
predominant throughout 

■ only at the highest level (Level 4) is there a significant 
need for strategic and reflective thinking 

■ strategic and reflective thinking are needed more 
in the key s kills of Working with Others, Improving 
Own Learning and Performance and Problem Solving 
than in Communication, Application of Numberand 
Information Technology. 



Here we summarise the results of meta-analytic 
research studies of thinking skills interventions. 

■ Thinking skill approaches with children and young 
people are usually very effective, especially if they 
are directed at metacognition, self-regulation and 
what we have called 'value-grounded thinking'. Their 
effectiveness is likelyto be greater if they are used for 
learner self-regulation ratherthan coming fully under 
teachercontrol. Well-focused interventions atthe 
cognitive level can also be very effective. These include 
interventions with a focus on experimental enquiry 
and the representation of ideas, as well as approaches 
to study support such as using cues and questions 

to aid retrieval. 

■ Unfortunately, we cannot yet be confident that equally 
impressive learning gains are achievable with post-16 
learners not in highereducation, as there are so few 
published studies. 

We believe that further research with, and use 
of, thinking skill frameworks will lead to a better 
understanding of how people think and learn. 

■ We think that teachers should be involved in the further 
development of theory-based thinking skill frameworks 
and in deciding howto use them. 

■ We recommend that practical applications of the 
new integrated framework presented in this report 
should be explored and evaluated. 

■ The interplay between theoryand practice in the 
development of good thinking in post-16 learners is 
a priority area forfuture research. 

■ We need to know more about which pedagogical 
interventions improve thinking in teachers and students 
and in which contexts. 

■ We need to know more about the advantages and 
disadvantages of encouraging creative diversity in 
thinking and in approaches to teaching and learning. 

■ We need to know more about the interpersonal aspects 
of thinking and learning, and especially about how 

to create and sustain productive learning communities, 
using face-to-face and distance-learning approaches. 

■ Using thinking skill frameworks, we need to develop 
new authentic, dynamic and ecologically relevant forms 
of assessment and evaluation. 
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We conclude by making a set of recommendations 
for policy-makers. 

■ The aims and objectives of post-16 education and 
training should be informed by explicitly stated values. 

■ Consideration should be given to developing new 
programmes focused on the study of human thinking, 
learning and behaviour. 

■ Courses and qualifications in which the demands 
on thinking and learning skills are incomplete, 
unbalanced orfragmented should be modified 
byapplying a theoretical framework of thinking 
and learning. 

■ Teachertraining courses should include a more solid 
grounding in theories of thinking and learning than they 
do at present. 

■'Community of enquiry’ and 'problem-based learning' 
approaches should be evaluated in the context of 
teachertraining. 

■ Dialogue and group discussion about how people think 
and learn should be available to all college students at 
the start of their studies and on a regular basis. 

■ Strategic thinking and reflection should form part of all 
education and training. 

■ Productive, strategic and reflective thinking - and 
not simply the reproduction and use of information 
to demonstrate understanding - should be assessed 
atall levels of education and training. 

■ Practitionerenquirythrough action research should 
be directed at finding ways of enabling people to build 
on previous learning and to make use of their skills 
and abilities in more areas of theirlives. 

■ Research and evaluation should be seen as 
collaborative activities, involving all stakeholders, 
not least the learners themselves. 
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Section 1 I 



Introduction 



1.1 

Aims 

In this report, we aim to review and advance 
knowledge of systematic ways of classifying thinking 
skills. We concentrate on taxonomies and theory-based 
frameworks, in the belief that studying and using them 
will promote better understanding of how people think 
and learn atthe age of 16 orabove. Ouroverarching 
interest is in how that understanding can inform 
instructional design, course and lesson planning, 
teaching, learning and assessment. In particular, we 
aim to identify: 

■ key principles on which teaching approaches designed 
to develop thinking skills depend 

■ helpful ways forteachers and learners to classifyand 
talk aboutthinking skills. 

1.2 

Scope 

While there are some theorists who have specifically 
addressed thinking skills in teenagers and adults, most 
classificatory systems applyacross the lifespan and in 
a wide variety of contexts. Some are intended to apply 
to the entire domain of thinking, some to only part 
of it, and some are part of a more broadly conceived 
theoretical framework for understanding individual 
differences and contextualised processes of thought 
and learning. Here we evaluate 35 frameworks which 
offer the prospect of improving understanding and 
practice in the post-16 context. We use the generic term 
'framework' to referto a taxonomy, a theory or a model. 

Because thinking is a human activity which involves 
cognition (knowing), affect (feeling) and conation 
(wanting and willing), we have not excluded from 
consideration frameworks which extend beyond the 
cognitive domain. 



1.3 

The research team 

The research team included members conversant 
with educational, philosophical and psychological 
discourse - the three main disciplines represented in 
the literature on thinking skills. One team member was 
involved in training college teachers, using a community 
of enquiry approach. The team also included people who 
are knowledgeable about multicultural issues, social 
exclusion and individual differences in learners. We 
drew on the experience of practitioners and academics 
in a number of countries. In addition, we received 
feedback and guidance from a regional advisory group 
and a steering group acting on behalf of the Learning 
and Skills Research Centre (LSRC). 

1.4 

Rationale 

Empirical work on teaching and assessing thinking 
provides a sound evidence base fortheory building and 
testing. There is a good deal of evidence to support 
the following points. 

■ There are broad and narrow thinking skills, as well as 
general psychological factors which affect how people 
think and learn. 

■ Thinking skills can be developed by means of teaching 
practices. 

■ Learning progress can be encouraged by thinking skills 
approaches. 

■ The value of thinking skills approaches stems from 
attention given to: 

■ pedagogy emphasising learnerengagement - 

in particular beliefs and feelings that help determine 
the motivation to learn 

■ metacognitive knowledge (knowledge of one's cognitive 
functioning, including knowledge developed through 
reflection) 

■ the strategic management of thinking and learning 
through self-regulation (involving planning, conscious 
direction, monitoring and evaluation). 

Education and training are purposeful activities 
through which participants can develop knowledge, 
skills and personal qualities. If a broad and 
balanced set of principles is used in the planning 
and delivery of education and training, processes and 
outcomes can be evaluated with reference to those 
principles. However, rational approaches to the 
planning, implementation and evaluation of education 
and training become ineffective where there is 
a kaleidoscopic arrayof ad hoc ideas and practices, 
influenced by academic or pedagogic fashion and 
commercial interests. 




Each of the many existing frameworks for understanding 
thinking is based (explicitly or implicitly) on particular 
value judgements about what is important in life or 
in lifelong learning. In a pluralistic society, with open 
access to information and with few restrictions on 
thinking and learning, it is highly unlikelythat anysingle 
coherent framework for developing thinking in post-16 
learners will prove adequate. We have worked on 
the assumption that there is value in a diversity of 
approaches, which suggests that educators should 
be encouraged to select frameworks which are fit for 
different purposes (eg improving communication skills, 
planning a vocational training course, developing 
problem-solving skills, encouraging creativity in product 
development, developing reflective judgement about 
current issues of global concern). 

Ourapproach has been to bring some orderto a highly 
complexfield, by identifying common features as well 
as differences between frameworks. We have focused 
on the organising principles on which the frameworks 
are based, noting that some emphasise progression 
in complexity of thought while others make qualitative 
distinctions between different kinds of thinking. 

In each evaluation, we applied criteria grouped under 
the following headings: 

■ scope and intended use 

■ underpinning values and theories 

■ practical relevance. 

1.5 

Context 

There is currently unprecedented political interest 
in the impact of education and training on social and 
economic development. The perceived need to improve 
standards, to widen access to higher education, and 
to minimise social exclusion (eg through the Skills for 
Life strategy and by participation in lifelong learning) is 
being addressed through many government initiatives. 
As in manyother countries, there is in England a 
politically driven preoccupation with testing and with 
other measurable educational outcomes. Policy-makers 
favour target setting and have, over the last 20 years, 
increasinglysoughtto prescribe curricula and methods 
of teaching and assessment, believing that this will 
lead to improved performance. Although the evidence 
base for what politicians, government-related agencies 
and inspectorates claim to be good practice is not 
always made explicit, government has made significant 
moves towards evidence-based policyand practice 
in education and in otherpublic services. 



Research evidence about the value of teaching 
thinking skills was summarised by McGuinness (1999), 
and thinking skills now form a recognised part of the 
National Curriculum in schools in England and Wales. 

In Qualifications and Curriculum Authority (QCA) 
documents, thinking skills are conceptualised under 
five headings: 

■ information processing 

■ reasoning 

■ enquiry 

■ creativity 

■ evaluation. 

Although these headings do not correspond with 
those used in an identifiable taxonomy ortheoretical 
framework, they are clearly broad in scope. We shall 
revisit the QCA categories in Section 4.3. 

Using these QCA categories, we identified the thinking 
skills needed to achieve National Curriculum learning 
objectives at Keystage 4 (14-16 years), limiting 
ourselves to subject areas where the objectives are 
available in electronic form. We found thatthinking 
skills feature explicitly in the following National 
Curriculum subjects (listed here in rank orderof word 
count, with mathematics having the largest number 
of objectives which make demands on thinking skills): 

■ mathematics 

■ science 

■ personal, social and health education 

■ citizenship 

■ physical education 

■ design and technology. 

Of course, broad thinking skills such as recalling 
prior knowledge, constructing an understanding 
and using knowledge are present in all areas of the 
National Curriculum. 

In post-16 education and training, ourimpression 
is that although there are many opportunities for 
developing thinking (notably through key skills 2 as well 
as in subject areas), relatively little has been done to 
embed thinking skills in pedagogy and in assessment. 
We see only a modest emphasis on thinking skills in 
the Adult Literacy Core Curriculum (Basic Skills Agency 
2001). Soden and Pithers (2001) found thatthis 
is equallytrue of manyFE college courses which lead 
to degrees. Because students have to understand 
and use a good deal of discipline-specific knowledge 
forexamination purposes, they relyon close direction 
from lecturers. 



2 

In England, key s kills are assessed in post-16 learners by tests and 
portfolios. There are six key skills specifications: Communication, 
Application of Number, Information Technology, Working with Others, 
Improving Own Learning and Performance, and Problem Solving. 
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There was very little evidence that students were 
gaining the experience of processing and interrogating 
knowledge independently which is associated with 
learning to think critically. 

(Soden and Pithers 2001,102) 

The same lack of attention to critical thinking by 
learners in adult education is apparent from an 
examination of the American Association for Adult 
and Continuing Education's (AAACE) Handbook of adult 
and continuing education (Wilson and Hayes 2000), 
where 'thinking' does not feature at all in the index and 
where the discourse of social constructivism is used 
to express ideas and ideals rather than to describe 
actual communities of learners. 

1.6 

Implications of contextual issues for the evaluation 

According to Andrich (2002), outcomes-based 
education has drawn heavilyon the pioneering 
work of Bloom. Bloom and his colleagues created their 
taxonomy of educational objectives (cognitive domain) 
because they wanted to classify intended behaviours 
'related to mental acts or thinking’, occurring 'as a 
result of educational experiences’ (1956, 12). Bloom's 
taxonomy, which has been more influential than any 
other, was originally designed in order to categorise 
examination questions at college level. Although the 
way in which thinking skills have been embedded 
in the Key stage 4 National Curriculum is not based 
on a single theoretical framework, there are clear links 
with the work of Bloom. Accordingly, we have given 
particular attention to Anderson and Krathwohl’s 
revision of Bloom’s taxonomy (2001). 

A simple framework, like the one used in the Key 
stage 4 National Curriculum, is iikelyto be more useful 
to teachers than a more complicated one, because 
it can be more easily grasped and applied in a person’s 
working memory. Anotheradvantage of the National 
Curriculum framework is that it does not contain 
specialised technical terms, making it accessible to 
all learners. We therefore looked forframeworks which 
are compatible with those already in use and which 
use relatively fa milia r terminology. We take seriously 
the work of Corson (1995), who showed thatthe use 
of language outside the vocabulary range of everyday 
speech limits access to learning by introducing 
a socio-cultural bias in favourof learners from more 
highly educated backgrounds. 



Believing, as government does, in the need to base 
practice on sound empirical evidence, we looked at: 

(a) factor analyses of cognitive abilities and skills; 
and (b) meta-analyses of the benefits attributable to 
pedagogical approaches based on various conceptions 
of thinking skills. We also noted the power of generic 
problem-solving tools (such as Alts huller's TRIZ; see 
Appendix 2) to help produce inventive solutions which 
draw on knowledge outside the area of application. 

We find the weight of evidence about the meaning, 
structure and operation of generic thinking skills 
compelling. From an empirical viewpoint, McPeck’s 
selectively supported argument (1981) that there 
is no such thing as a generic thinking skill looks very 
much like word-play. 

We are aware that practitioners are subject to many 
pressures to change and improve their practice, 
and that there is a tension between traditional teacher- 
directed pedagogyand the orientation towards learner 
empowerment with which thinking skills approaches 
are often associated. In a crowded, externally driven 
curriculum, will teachers have the time and thinking 
space to accommodate yet another innovation? 
Because we were unclear as a team as to how teachers 
and post-16 learners view thinking skills, we set up 
three consultative meetings with college lecturers and 
followed this up with a questionnaire. We also learned 
about an LSDA-funded pilot project at Gateshead 
College in which a communityof enquiry approach was 
introduced to various course groups at Key S kills Levels 
1 and 2 (Duffy 2003). These experiences helped us 
to evaluate frameworks forclassifying thinking skills 
in terms of theirapplicability in the post-16 sector. 

1.7 

Method 

Ourfirst step was to carryout a comprehensive 
and systematic literature search of electronic and 
paper-based sources. Over400 articles and books 
were identified as relevant. We also found a large 
number of useful websites, many of which are gateways 
to furthersources. We have listed many of these at 
www.ncl.ac.uk/ecls/about/ resource/ thinking, htm. 

We identified 55 thinking skill frameworks by reading 
our source material, checking references or learning 
about them at conferences. We prepared brief 
summaries of each and located them on a chart within 
four 'family groups’ and by decade of publication 
(see Appendix 1). 



We decided not to evaluate 20 of the frameworks, 
rejecting those which differonlyin minor detail from 
more established conceptualisations, those [like 
Piaget's (1952) genetic epistemology] which have been 
developed and extended by othertheorists specifically 
for post-16 use, and those which have little relevance 
forthe post-16 sector. 

In orderto ensure a consistent approach to evaluation, 
team members prepared and discussed briefing 
papers on core concepts and two working papers on: 

(a) evaluation criteria; and (b) the relevance of key 
taxonomies for post-16 learners. Team members wrote 
descriptions and evaluations, referring to established 
models and receiving feedback from colleagues. 

We used an argument mapping software tool called 
Reasonable (van Gelder 2000) to help us to support, 
qualify and compare the strength of claims formulated 
fororagainst each framework. 

The team then had to decide which frameworks to 
include in the main text of this report, bearing in mind 
the project aims stated in Section 1.1 and the criteria 
referred to in Section 1.4 above. We also wanted to 
ensure a representative coverage of the field and 
to avoid undue overlap. This was done through a voting 
procedure, with votes supported by reasons, and 
by group discussion in which we again drew on our 
evaluation criteria. 

Aftera preliminary evaluation midwaythrough the 
project, two frameworks - those of Anderson and 
Krathwohl (2001) and Marzano (2001a, 2001b) - stood 
out as being strongly grounded in theory and practice. 
Theyaddress instructional design in similarways, 
but Marzano’s new taxonomy of educational objectives 
is broader in scope, well grounded in psychological 
theoryand supported bya large-scale meta-analysis 
of research into educational interventions. The two 
frameworks are compatible with others which focus 
in greaterdetail on, forexample, critical and 
creative thinking. 

In orderto test out our ideas aboutthe practical 
relevance of these and otherframeworks for post-16 
teachers and learners, we asked 37 FE teachers to rate 
the importance of 69 different thinking skills found in 
the frameworks we had evaluated. We established that 
both the Anderson and Krathwohl and the higher-level 
Marzano categories correspond quite closely with 
how practitioners alreadythink a bout thinking skills. 

We were able to use the results to help create a new 
framework. This is compatible with manyothers, 
including the National Curriculum categories, but is 
especially relevant for post-16 teachers and learners 
because of its emphasis on orchestrated productive 
thinking 3 skills and on strategic and reflective thinking. 



3 

By productive thinking, we mean purposive thinking which results 
in a new and valued outcome that is more than basic understanding. 
Fora fuller exposition, see Section 3.4. 



We tested out a simplified version of the new framework 
by using it to classify key skills knowledge objectives, 
finding that this was both feasible and illuminating. 

We were then in a position to assemble arguments 
to support recommendations for general and specific 
uses of certain frameworks. 

1.8 

How the report is organised 

The research is presented in Sections 2-5. Section 6 
provides a summaryof what is known and indicates 
the need for further research in the post-16 sector. 

The following section outlines will help with navigation. 

■ Section 2 

Surveying and mapping the field 

Here we clarify the key concepts which are fundamental 
to an understanding of the field. We do this by referring 
to widely accepted definitions and views, and by 
providing ourown description of how 'teaching thinking 
skills’ is widely understood in educational discourse. In 
orderto facilitate understanding, we clarify the meaning 
of all key concepts, either in the text or in a Glossary. 

We then explain why and how we grouped the 
55 frameworks into 'families'. After briefly introducing 
each 'family group’, we list the frameworks that belong 
to it and are evaluated later in the report. 

■ Section 3 

Thinking skill frameworks evaluated 

Nine full accounts and evaluations are included 
in Section 3, again grouped byfamilyand with some 
mention of family resemblances, special features 
and overlaps. Some readers will wantto refer to 
Appendix 2 at this point in orderto considerframeworks 
not included in the main text. 

■ Section 4 

M aking sense of thinking and learning 

Here we pull outthe principles used to establish 
categories in all the frameworks we evaluated. 

We lookcloselyatthe frameworks in which atleast 
four different kinds of principle are used. We identify 
common features in a number of these frameworks, 
and use them as the basis forconstructing a new 
integrated framework. We show how we began with 
a six-level categorisation, which we then validated and 
further developed by finding out how college teachers 
mentally group thinking skills (see Section 1.7 above). 

We end the section by identifying three frameworks 
which best represent the field, are compatible with 
our new integrated framework and have the most 
to offer in terms of general use in the post-16 sector. 
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■ Section 5 

Some ideas for using the frameworks 

We begin this section by reporting on the key skills 
study referred to in Section 1.7 above. We then bring 
together the findings of ourevaluations to assess 
the potential value of a number of thinking skill 
frameworks fora varietyof purposes in post-16 
education and training. Taking account of contextual 
factors and present levels of knowledge about thinking 
skills and pedagogy, we identify a range of possibilities 
for practical application and development. 

■ Section 6 

Summary of what is known and the need for 
further research 

We summarise what we believe to be well-founded 
and less well-founded knowledge about thinking skill 
frameworks and aboutthinking, learning and instruction 
in a broadersense. We then identify areas where new 
research is needed, beginning with teacher and learner 
understanding and the practical application of central 
concepts, frameworks and theories. We end by 
making recommendations about how the well-founded 
knowledge should be applied and extended in the 
post-16 context. 
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Surveying and mapping the field 



2.1 

Why classify? 

Trying to understand how people think and learn is 
in some ways an impossible challenge, since we can 
only try to understand these things by using the very 
processes that we do notfullyunderstand. In such 
circumstances, however; choices are available. 

We can choose to focus on measurable aspects 
of human behaviour rather than on lived experience; 
or we can resort to metaphors which have personal or 
group appeal; or we can do what scientists have often 
done when entering a new and complex field - look 
for patterns. All three approaches are evident in the 
theoretical frameworks and taxonomic approaches 
to thinking and learning that we have studied, and 
they all involve classification. Moreover they all 
result in accounts which focus on certain aspects 
at the expense of others, since the human mind 
can onlyoperate consciously with limited amounts 
of information, especially when communicating 
with others. Minds are intrigued bythe richness 
of immediate experience as well as by patterns and 
the power of communicable ideas. 

2.2 

An inclusive approach 

Cognitive psychologists typically study thinking in 
other people - a third-person perspective in which 
the metaphor of the brain as a computer has been 
dominant. First-person introspective accounts of 
thinking have a different feel about them, since we 
all have the impression that we can consciously control 
ourthoughts and actions. We experience wanting, will, 
effort and emotion in a holistic manner as we think, 
and it is only through subsequent analytical reflection 
that we can viewthese aspects dispassionately. 
Indeed, a case can be made that while we are thinking 
(with our attention focused on certain elements), 
we are not aware of the thinking process itself (much 
of which is unconscious). It is onlyafterthe event 
that we can reflect on the products of ourthinking 
and to a certain extent, reconstruct and analyse the 
process. Like Velmans (2000), we have taken the view 
that first-person and third-person accounts of thinking 
are complementary and that one cannot be reduced 
to the other. 



A teacher has a third-person perspective on the 
learner’s thinking and can only make inferences about 
iton the basis of whatthe learner does. Some earlier 
approaches to instructional design focused on precisely 
formulated, externally imposed behavioural objectives 
in place of goals which learners set forthemselves or 
agree with others. Goal setting and ways of monitoring 
and evaluating progress are under-researched in the 
post-16 field, but it is likelythat an inclusive and flexible 
approach will prove most effective. First-person goal 
setting maybe desirable in some contexts and with 
certain types of content; whereas group negotiation 
of goals maybe preferred in other contexts; and 
teacher- or other externally driven instruction may 
be most effective in yet other contexts, where mastery 
learning and accurate performance is expected. This 
argument applies just as much to the development 
of thinking skills as to anyotherkind of learning. 

2.3 

What are thinking skills? 

When there is some uncertaintythat a satisfactory 
end is achievable, it is useful to think. The word 
'thinking' is usually used to mean a consciously 
goal-directed process, such as remembering, forming 
concepts, planning what to do and say, imagining 
situations, reasoning, solving problems, considering 
opinions, making decisions and judgements, and 
generating new perspectives. Controlling what we call 
conscious thought presumably influences unconscious 
processes, some of them internalised and automatic 
as a result of practice. These processes also help to 
generate desired thinking products. When we refer 
to thinking skills, we do not exclude the unconscious 
elements, anymore than when we refer to the skills 
of an Olympic athlete. 

'Skill' commonly means 'expertness’, 'practical ability’ or 
'facility in doing something’ (Oxford English Dictionary). 
The term refers to what you have to know and what you 
have to be able to do to succeed in a task. The concept 
of 'skill' overlaps with 'ability', but the term 'skill' is more 
often used to refer to specific rather than to general 
areas of performance. Having a skill implies that most 
performances are of a high standard and are adapted 
to the requirements of particular situations. 

On this basis, 'thinking skill' means expertness, 
practical ability orfacility in the process orprocesses 
of thinking (processes that occurspontaneously 
or naturally, or which are acquired through learning 
and practice). Whetherornota particular instance 
of thinking is skilled cannot be judged by others 
without a great deal of contextual knowledge, including 
knowledge of the prior experiences of the individual 
or group. 




In educational discourse, 'teaching thinking' or 
'teaching thinking skills’ is often used to referto 
pedagogic approaches through which specific strategies 
and procedures may be taught and used by learners 
in a controlled, conscious wayto make theirthinking 
more effective. These strategies and procedures 
maybe whatsome use spontaneously and/ orthey may 
be otherwise contrived. Many such skills and abilities 
have been suggested - specific, broad, or general 
in nature. Ashman and Conway (1997) conclude that 
thinking skills programmes typically involve six related 
types of thinking: 

■ metacognition 

■ criticalthinking 

■ creative thinking 

■ cognitive processes (suchas problemsolvingand 
decision making) 

■ corethinkingskills (such as representation and 
summarising) 

■ understanding the role of content knowledge. 

We conceptualise 'thinking skills approaches’ as 
courses ororganised activities which identify for 
learners translatable 4 mental processes and / or which 
require learners to plan, describe and evaluate their 
thinking and learning. This usage of the term 'thinking 
skills' implies that there are activities that can induce 
processes which produce desired mental products. 

It is underpinned bya judgement that thinking can 
be improved with practice and/orteaching (ie ability, 
orat least performance, can be improved). It also 
implies the use of mental processes to plan, describe 
and evaluate thinking and learning. One way of looking 
at this metacognitive aspect is to considerthinking 
skills as ways of managing attention and working 
memory so that conscious and unconscious processes 
togetherare more likelyto produce desired outcomes 
(Newton 2000). 



4 

'Translatable', unlike 'transferable', does not implythat exactly the same 
process is used in different contexts. It suggests that the process or 
processes are adaptable, but still retain a 'family resemblance' across 
situations, so that the meaning of a process like 'summarising' or 
'evaluating' is preserved. 



2.4 

What is metacognition? 

Perry (1970) spoke about 'meta-reason' and 
'meta-thought', but the coining of the term 
'metacognition' is usually attributed to Fla veil 
(1976, 232): 

Metacognition refers to one’s knowledge concerning 
one’s own cognitive processes and products or 
anything related to them ... Forexample, I am engaging 
in metacognition (metamemory, metalearning, 
metaattention, metalanguage, or whatever) if I notice 
that I am having more trouble learning A than B; if it 
strikes me that I should double-check C before accepting 
it as a fact; ... if I sense that I had better make a note 
of D because I mayforget it; ... Metacognition refers, 
among otherthings, to the active monitoring and 
consequent regulation and orchestration of these 
processes ... usually in the service of some concrete 
goal or objective. 

Metacognition involves two majordimensions 
(Boekaerts and Simons 1993). First, it involves 
an awareness of one’s own cognitive functioning 
(metacognitive knowledge); and second, the planning, 
monitoring and evaluation of one’s thinking and 
learning. It is important to note that 'metacognition' 
is used narrowly bysome (simplyto mean 'thinking 
a bout thinking') and more broadly by others (to include 
the conscious self-regulation of thinking and learning). 

Are there metacognitive thinking skills? Swartz and 
Parks (1994, 520) believe that there are, as they 
write about 'skillful metacognition' and provide a list 
of the components. But what does it mean to say that 
a person is skilled at (forexample) goal specification, 
process specification, process monitoring ordisposition 
monitoring? Within a specific context or domain, 
we can understand that, forexample, an air-traffic 
controllercan be skilled at monitoring his orher 
level of vigilance. Flowever, when we think about 
cross-domain competence in (say) goal setting, we 
mayor may not have sufficient knowledge to be able 
to assess a person’s metacognitive skills; and it may 
be equally, if not more, appropriate to think in terms 
of a well-established disposition to plan systematically. 
So when Mayer(1998) describes the ability to control 
and monitorcognitive processes as a 'metas kill', 
we understand that this author is referring to very 
much the same capacities as Perkins, J ay and Tishman 
(1993, 6) when they write about 'the disposition to 
be metacognitive'. 
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2.5 

What is self-regulation? 

Most researchers agree that self-regulation is 
a systematic process involving the setting of personal 
goals and the channelling of one's behaviourtowards 
theirachievement. Self-regulation involves cognitive, 
motivational, affective and behavioural components that 
enable individuals to adjusttheiractions and/ortheir 
goals in orderto achieve desired results in changing 
environmental circumstances. Such adjustment 
presupposes metacognition and cannottake place 
withoutsome conscious awareness of the ongoing 
thinking and learning processes. It should be noted 
that self-regulation is widely considered to be highly 
context-specific. 

According to Schunk and Ertmer (2000), self-regulation 
comprises such processes as: 

■ setting goals forlearning 

■ attending to and concentratingon instruction 

■ using effective strategies to organise, code and rehearse 
information to be remembered 

■ establishinga productive work environment 

■ using resources effectively 

■ monitoring performance 

■ managing time effectively 

■ seekingassistance when needed 

■ holding positive beliefs a bout one’s capabilities, 

the value of learning, the factors influencing learning, 
and the anticipated outcomes of actions 

■ experiencing pride and satisfaction with one’s efforts. 

Classroom interventions based upon theories 
of self-regulation emphasise the importance of helping 
students to develop a positive orientation to learning 
and a belief thattheyare capable of succeeding if 
they work hard and use appropriate strategies. While 
these elements also feature in manythinking skills 
programmes, theyare sometimes more implicit 
than explicit. 



2.6 

What is critical thinking? 

The literature on critical thinking is extensive. The 
term is used in different ways and has developed over 
time (eg see the review by Pithers and Soden 2000). 

In the US, critical thinking is often considered to be 
synonymous with thinking skills. 

Descriptive definitions of critical thinking tend to 
be psychological in origin. They specify cognitive 
skills and the mental processes involved in different 
aspects of thinking, often equating them with the 
higher-order categories of Bloom's taxonomy. Implicit 
in this approach is that being good at critical thinking 
is being proficient at particular mental processes 
such as analysing, inferring, evaluating. By contrast, 
philosophers argue fora normative definition. 

Bythis theymean thatcriticai thinking is inextricably 
connected with values and essentially means 'good 
thinking’. From this perspective, purely descriptive 
accounts omit the central issue of the quality 
of the thinking. 

So, forexample, consider making a decision about 
whetherornot to adopta local recycling scheme. 

From a descriptive perspective, critical thinking involves 
analysing the issue, generating possible resolutions, 
evaluating these and synthesising the information 
to reach a decision. Flowever, the decision may not be 
a good one, as the issue may have been analysed from 
superficial perspectives (residents do not have space 
fora second rubbish bin ormaybe confused about 
which bin to use), orthe options may have been 
evaluated from a biased perspective (the local factory 
which makes recycling bins argues foreach household 
to have a bin foreach kind of recyclable rubbish). 

One of the influential and enduring definitions of critical 
thinking comes from Ennis (1985, 45): 'reflective and 
reasonable thinking that is focused on deciding what 
to believe or do'. Ennis believes thatcriticai thinking 
depends on two overarching dispositions: caring to 
get it right to the extent possible, and caring to present 
positions honestlyand clearly (Ennis 1998). It also 
depends on the process of evaluation (applying criteria 
to judge possible answers). 

The conceptualisation and teaching of critical thinking 
were greatly influenced in the US by the publication 
of The Delphi Report (Facione 1990). This sought 
to provide a consensual account of the nature and 
dynamics of critical thinking arrived at by a panel 
of leading figures in the field, including Costa, Ennis, 
Lipman and Paul (reviews of the theories of Ennis, 

Paul and Lipman are presented in Section 3.4.1, 

Section 3.4.3 and Appendix2 respectively). 



The Delphi Report identifies elements of critical thinking 
in terms of the following skills and sub-skills: 



Skill 


Sub-skill 


Interpretation: 


categorisation 
decoding significance 
clarifying meaning 


Analysis: 


examining ideas 
identifying arguments 
analysing arguments 


Evaluation: 


assessing claims 
assessing arguments 


Inference: 


querying evidence 
conjecturing alternatives 
drawing conclusions 


Explanation: 


stating results 
justifying procedures 
presenting arguments 


Self-regulation: 


self-examination 

self-correction 



An important element of the critical thinking approach 
is its emphasis upon the close relationship between 
abilities and dispositions. The Delphi panel emphasised 
the importance of dispositions in the self-examination 
and self-corrective aspects of critical thinking and 
argued that the appropriate exercising of each cognitive 
skill could be related to the cognitive disposition 
to do so. 

2.7 

Classifying the frameworks 

As we became increasingly familiar with the thinking 
skill frameworks, fourmain 'family groups' emerged. 
These are: 

1 

models and theories of personality, thought and 
learning (the all-embracing family) 

2 

models and theories of instructional design 
(the designer family) 

3 

models and theories of critical or productive thinking 
(the higher-order family) 

4 

models and theories of cognitive structure and/or 
cognitive development (the intellect family). 



We decided to place each framework in one of the 
fourgroups, findingthatthis helped us to understand 
how the field has developed and also made it easier 
to evaluate similarframeworks more consistently. 
Although there is a good deal of overlap between 
the groups, the completed classification brings out 
differences between families as well as common 
features and underlying principles which they 
sometimes share. 

The resulting chart (see Appendix 1) gives brief 
descriptions and shows which frameworks deal with 
the cognitive domain, the affective domain, conation 
and metacognition. It also indicates the academic 
discipline of each framework’s main authorand whether 
ornot it was designed foreducational use. As can 
be seen from the chart, most of the all-embracing 
frameworks were created in recent years, three 
of the seven members appearing in the present century. 
We think of them as all-embracing in that they deal 
with emotional and motivational influences on thinking 
and learning as well as with the structure of cognition. 
'All-embracing' is not meant to suggest that they 
encompass all the main features of the otherfamilies. 

We placed a framework in the instructional design 
family if it was created primarilyforthat purpose 
and has a cognitive ratherthan a more comprehensive 
theoretical emphasis. The criterion forassigning 
a framework to the critical/ productive thinking family 
was whether it is concerned with higher-order processes 
leading to a judgement or decision. Dispositions, 
as well as abilities and skills, feature in manycritical 
thinking frameworks. The members of the fourth 
family- explanatory models of cognitive structure 
and/ ordevelopment - are of two main types: some 
simply specify sets of intellectual aptitudes orabilities, 
and others describe stages of progression towards 
more complex or mature ways of thinking. 

To a certain extent, groups 2, 3 and 4 can be seen 
to reflect the concerns and interests of different 
disciplines: education (instructional design), philosophy 
(critical thinking) and psychology (mental activity 
and development). 

2.8 

Thefamilygroups 

The 35 frameworks evaluated in this report are listed 
below in approximate chronological order, with a brief 
introduction to each family group. The titles of the 
nine frameworks which we evaluate in the main text 
of the report are untinted. Descriptions and summary 
evaluations of the other 26 frameworks can be found 
in Appendix2. 
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2 . 8.1 

Group 1 : all-embracing frameworks, 
covering personality, thought and learning 

While all five authors within this group seek to provide 
a comprehensive account of thinking and learning in 
which emotions and beliefs play a part, only Hauenstein 
(1998), J onassen and Tessmer (1996/ 97) and Marzano 
(2001a, 2001b) classify educational goals within 
and beyond the cognitive domain. Sternberg's model 
(2001) was not designed forthis purpose, although 
it does have significant pedagogical implications. 
Romiszowski’s writing (1981) has a broad theoretical 
scope as well as a focus on course planning and 
curriculum design. His analysis of skills in the 
form of a learning cycle was not designed to replace 
existing taxonomies of educational objectives, but 
as a 'conceptual tool to analyse the causes of poor 
performance' (1981, 257). 

1980-1989 

Romiszowski's analysis of knowledge and skills 

Romiszowski distinguishes between reproductive 
and productive learning in fourskill domains: cognitive, 
psychomotor, reactive and interactive. He identifies 
12 abilities which maybe used in perception, recall, 
planning and performance. 

1990-1999 

Hauenstein’s conceptual framework for 
educational objectives 

Acquisition, assimilation, adaptation, performance 
and aspiration are successive levels of learning 
in the cognitive, affective and psychomotor domains. 

At each level, and within each domain, Hauenstein 
identifies processes which help to build understanding, 
skills and dispositions. 

J onassen and Tessmer's taxonomy 
of learning outcomes 
(see Section 3.2.1 ) 

The major categories in this taxonomy are: 
declarative knowledge, structural knowledge, 
cognitive component skills, situated problem solving, 
knowledge complexes, ampliative skills, self-knowledge, 
reflective self-knowledge, executive control, motivation 
(disposition) and attitude. 

Vermunt and Verloop's categorisation 
of learning activities 

The cognitive categories are: relating/ structuring, 
analysing, concretising/ applying, memorising/ 
rehearsing, critical processing, selecting. The affective 
categories cover motivation and the management of 
feelings. The regulative categories are an elaboration 
of 'plan-do- review'. 



2000 - 

Marzano’s new taxonomy of educational objectives 
(see Section 3.2.2 ) 

The self system examines the importance of new 
knowledge, efficacy (ability to learn) and emotions 
associated with knowledge and motivation. The 
metacognitive system specifies learning goals and 
monitors execution, clarity and accuracy. The cognitive 
system deals with retrieval, comprehension, analysis 
and knowledge utilisation. 

Sternberg’s model of abilities as developing 
expertise 

This model includes the analytical, creative 
and practical aspects of successful intelligence, 
metacognition, learning skills, knowledge, 
motivation and the influence of context. 

2 . 8.2 

Group 2 : instructional design frameworks 

Bloom's well-known taxonomy (1956) has been 
used throughout the world as a framework for 
designing instruction, and many similarities, if not 
direct influence, can been seen in the work of Ausubel 
and Robinson (1969), Gagne (1965, 1985), Hannah 
and Michaelis (1977), Stahl and Murphy (1981), and 
Anderson and Krathwohl (2001). All these authors 
provide frameworks forformulating and classifying 
educational goals in terms of the thinking and learning 
processes which can be inferred from observed 
behaviourortask performance. Anderson and 
Krathwohl (2001) explicitly revised Bloom's cognitive 
domain taxonomy. 

Asecond group of authors - Biggs and Collis (1991), 
Quellmalz (1987), Presseisen (1991, 2001), and 
Gouge and Yates (2002) - share an interest in designing 
instruction so as to develop higher-order thinking, and 
have built conceptual frameworks for understanding 
how thinking skills are orchestrated for purposes 
such as decision making, problem solving, critical and 
creative thinking. The focus here sometimes extends 
beyond the cognitive domain, and in most cases 
includes an account of metacognition. 

Feuerstein’s Instrumental Enrichment (IE) programme 
(1980) represents an individual approach to 
instructional design, in which practitioners base 
their work on a specific set of cognitive tasks and 
pedagogical principles. 



1950-1959 

B loom's taxonomy of educational objectives 
(cognitive domain) 

This framework is a wayof classifying educational goals 
in terms of complexity. The intellectual abilities and 
skills of comprehension, application, analysis, synthesis 
and evaluation are applied to and help build knowledge. 

Feuerstein's theory of mediated learning 
through Instrumental Enrichment 

Building on his belief in cognitive modifiability, 
Feuerstein developed the concept of a mediated 
learning experience in which the mediator uses 
prescribed tasks to promote thinking ratherthan 
rote learning. 

1960-1969 

Ausubel and Robinson’s six hierarchically 
ordered categories 

These are: representational learning, concept learning, 
propositional learning, application, problem solving 
and creativity. 

Gagne’s eighttypes of learning and five types 
of learned capability 

Gagne set out an eight-level hierarchy of learning types, 
with problem solving at the top. He also identified five 
domains of learning: motor s kills, verbal information, 
intellectual skills, cognitive strategies and attitudes. 

1970-1979 

Hannah and M ichaelis's comprehensive framework 

for instructional objectives 

The cognitive, psychomotor and affective domains 
are covered. Interpreting, comparing, classifying, 
generalising, inferring, analysing, synthesising, 
hypothesising, predicting and evaluating are listed 
as intellectual processes. 

Williams' model for developing thinking and 
feeling processes 

This three-dimensional cross-curricular model 
seeks to encourage creativity. Teachers can use 
18 teaching modes to promote fluency, flexibility, 
originality, elaboration, curiosity, risk taking, 
complexity and imagination. 



1980-1989 

Biggs and Collis's SOLO taxonomy 
(see Section 3.3.1 ) 

This is an assessment tool looking atthe structure 
of the observed learning outcome. Prestructural 
responses betray limited understanding compared 
with unistructural and multistructural responses. 
Relational and extended abstract responses are 
qualitatively superior. 

Quellmalz's framework of thinking skills 

This framework lists five cognitive processes 
(recall, analysis, comparison, inference/ interpretation 
and evaluation) and three metacognitive processes 
(planning, monitoring and reviewing/ revising). 

Stahl and M urphy's domain of cognition 
taxonomic system 

These authors set out a multi-stage model 
of information processing from preparation to 
generation. They also identify 2 1 cognitive processes 
(eg classifying, organising, selecting, utilising, 
verifying), which maybe used singly orin combinations 
at different levels. 

1990-1999 

Presseisen's models of essential, complex and 
metacognitive thinking skills 

Presseisen lists five basic processes which are used 
in problem solving, decision making, critical thinking 
and creative thinking. She also lists six metacognitive 
thinking skills involved in strategy selection, 
understanding and monitoring. 

2000 - 

Anderson and Krathwohl's revision 
of B loom's taxonomy 
(see Section 3.3.2 ) 

Bloom’s taxonomy (1956) has been refined and 
developed into a two-dimensional framework using six 
cognitive processes and four knowledge categories. 
There is an emphasis on aligning learning objectives 
with learning activities and assessment. 

Gouge and Yates’ ARTS Projecttaxonomies 

of arts reasoning and thinking skills 

A matrix of Piaget’s levels (concrete, concrete 
transitional and formal operational thinking) and 
reasoning skills is used to create educational 
objectives forthe visual arts, music and drama. 
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2.8.3 

Group 3 : frameworks for understanding critical 
and productive thinking 

This is a fairly homogeneous group where the 
main emphasis is on the quality of higher-order thinking 
and reasoning. Good thinking is aided by reflection 
and metacognition. All authors identify dispositions 
which they believe to be extremely important in 
the development of critical and/ or productive thinking, 
with the exception of Allen, Feezel and Kauffie 
(1967) (who limitthemselves to argument analysis) 
and Gubbins (1986) (whose purpose was to make 
a composite list of critical thinking skills fora 
school district). 

1950-1959 

Altshuller's TR I Z theory of inventive problem 
solving 

There are four main steps: problem definition; 
problem-solving tool selection; generating solutions; 
evaluating solutions. Aspecific problem is an 
instance of a generic problem which is solved when 
the appropriate generic solution is returned to 
a specific solution. 

1960-1969 

Allen, Feezel and Kauffie’s taxonomy of concepts 
and critical abilities related to the evaluation of 
verbal arguments 

Twelve abilities are involved in the recognition, analysis 
and evaluation of arguments. Truth claims depend 
on testimonyand reasons. People should not be misled 
by rhetoric orthe misuse of language. 

1970-1979 

Lipman's three modes of thinking and four main 
varieties of cognitive skill 

J udgement and reasoning can be strengthened through 
critical, creative and caring thinking. In education, 
the four major varieties of higher-order thinking relate 
to: enquiry, reasoning (preserving truth), information 
organising and translation (preserving meaning). 



1980-1989 

Baron's model of the good thinker 

The most important components of the model are 
the three conscious search processes - for goals, for 
possibilities and for evidence. Good thinking and the 
dispositions underlying it are to some extent teachable. 

Ennis's taxonomy of critical thinking 
dispositions and abilities 
(see Section 3.4.1) 

'Critical thinking is reasonable reflective thinking 
that is focused on whatto believe or do' (1985, 45). 

For Ennis, the basic areas of critical thinking are clarity, 
basis, inference and interaction. Fie lists 12 relevant 
dispositions and 15 abilities. 

Gubbins' matrix of thinking skills 

This is a composite list of 'core' critical thinking skills, 
based on other published lists. The skills are grouped 
underthe following headings: problem solving, decision 
making, inferences, divergent thinking, evaluating, 
philosophy and reasoning. 

Flalpern's reviews of critical thinking skills 
and dispositions 
(see Section 3.4.2) 

Flalpern's skill categories are: memory, thought and 
language; deductive reasoning; argument analysis; 
hypothesis testing; likelihood and uncertainty; decision 
making; problem solving; and creative thinking. 

She also lists six relevant dispositions. 

Paul's model of critical thinking 
(see Section 3.4.3) 

The model has four parts: elements of reasoning, 
standards of critical thinking, intellectual abilities 
and intellectual traits. The first three parts focus on 
what is essential to critical thinking and the fourth 
on what it is to be a critical thinker. 

1990-1999 

J ewell's reasoning taxonomy for gifted children 

Jewell's taxonomy has three fields: objectives of 
reasoning, reasoning strategies and reasoning 
dispositions. The disposition to adopt thinking about 
thinking (metacognition) as a habit is very important. 



2.8.4 

Group 4 : explanatory models of cognitive 
structure and/or cognitive development 

This family group is the most diverse and includes 
different approaches to analysing the concept 
of intelligence. Some theorists (Carroll 1993, Guilford 
and Gardner) categorise thinking and problem-solving 
processes in similarways in school-aged children and 
in adults, but have different views about genetic and 
environmental influences on the structure of intellect. 
Stage theorists argue for qualitative changes in thinking 
as the learnerdevelops orprogresses (Perry 1968, 

1970; Belenky et al. 1986; King and Kitchener 1981, 
1994; Koplowitz 1987). Pintrich (2000) focuses 
exclusively on metacognition and self-regulation. 

1960-1969 

Guilford's structure of intellect model 

This is a three-dimensional model in which five cognitive 
operations work with fourtypes of content to produce 
six types of product. The operations are: cognition, 
memory, divergent thinking, convergent thinking and 
evaluation. 

Perry's developmental scheme 

The scheme consists of nine positions which liberal arts 
college students take up as they progress in intellectual 
and ethical development. They move from the modifying 
of 'either or’ dualism to the realising of relativism and 
then to the evolving of commitments. 

1980-1989 

Belenky's 'women's ways of knowing' 
developmental model 

Women in adult education tended to progress from: 
silence (a reaction to authority), to received knowledge, 
to subjective knowledge, to procedural knowledge 
(including separate and connected knowing), and finally 
to constructed knowledge. 

Gardner's theory of multiple intelligences 

Gardner identifies nine kinds of intellectual ability: 
verbal/ linguistic, mathematical/ logical, musical, 
visual/ spatial, bodily/ kinaesthetic, interpersonal, 
intrapersonal, naturalist and existential. 

King and Kitchener's model of reflective judgement 
(see Section 3.5.1 ) 

This is a seven-stage model of progression in 
adolescent and adult reasoning. Assumptions about 
knowledge and strategies for solving ill-structured 
problems can move from pre-reflective through 
quasi-reflective to reflective stages. 

Koplowitz’s theory of adult cognitive development 

Koplowitz builds on Piaget's stage theory, but adds 
two postmodern stages beyond the formal operations 
stage - post-logical and unitarythinking. The stages 
reflect changes in how people understand causation, 
logic, relationships, problems, abstractions 
and boundaries. 



1990-1999 

Carroll's three-stratum theory of cognitive abilities 

This theory has a well-founded empirical basis 
forthinking of cognitive tasks as making demands 
on narrow and/ or broad abilities as well as on 
general intelligence. 

2000 - 

P intrich's general framework for 
self-regulated learning 
(see Section 3.5.2 ) 

Pintrich identifies four phases of self-regulation. 
Cognition, motivation/ affect, behaviourand context 
can be regulated by: (1) forethought, planning and 
activation, (2) monitoring, (3) control, and (4) reaction 
and reflection. 
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Thinking skill frameworks evaluated 



3.1 

Evaluation criteria 

We established a set of criteria in orderto evaluate 
each framework in terms of the purpose(s) for which 
it was devised and the use(s) to which it has been 
put. We decided to organise our evaluation reports 
usingfourmain headings: description and intended 
use, scope and structure, theoryand values, and 
communicability and practical relevance. 

The following aspects were taken into account: 

■ description and intended use 

■ nature and function: 

taxonomy/ framework/ model/ map/ list 

■ stated purpose 

■ scope and structure 

■ the domains and/ orsub-domains addressed 

■ the principle orprinciples used in constructing 
the framework 

■ structural complexity and level of detail 

■ the thinking skill categories 

■ the thinking skill elements 

■ how well the domains and/ orsub-domains are covered 

■ extent to which categories overlap 

■ overall coherence 

■ distinctiveness 

■ theoryand values 

■ justification forchoice of underlying principles 

■ explanatory power 

■ compatibility with similarsystems 

■ consistency with well-supported theories 

■ pedagogical stance (if any) 

■ values: explicit/ non-explicit; descriptive/ prescriptive 

■ communicability and practical relevance 

■ clarity of formulation 

■ accessibilityforteachers and learners 

■ actual and potential areas of application 

■ implications forunderstandingteachingand learning 

■ match with a range of post-16 learning activities 

■ implications for practice in post-16 learning contexts 

■ actual and potential use in research. 



We applied ourevaluation criteria to systems 
of classification which are variously described 
as taxonomies, conceptual frameworks or models. 

Some strict criteria, such as the need to have 
comprehensive and non-overlapping categories, 
and to apply classificatory principles in an entirely 
consistent mannerare applicable to taxonomies, 
but less so to frameworks and models. We found some 
cases where we felt that what the author describes 
as a taxonomy is reallya frameworkora model, and 
in such cases we were more interested in clarity 
and heuristic value than in theoretical elegance. 

3.2 

Evaluation of all-embracing frameworks 

The two broad frameworks considered here offer ways 
of classifying the knowledge and skills required to 
achieve educational objectives. Three otherframeworks 
in which learning features as much as thinking can be 
found in Appendix2. Theywere devised by Romiszowski 
(1981), Hauenstein (1998) and Sternberg (2001). 

3.2.1 

J onassen and Tessmer's taxonomy of learning 
outcomes 

Description and intended use 

This taxonomy was created byj onassen and Tessmer 
(1996/ 97) 'for the development and evaluation 
of computer-based learning systems for higher order 
thinking skills’ (Jonassen, Tessmerand Hannum 1999, 
30). However, the authors believe that it has wider uses 
and regard it as an evolving entity. They argue that 
current research taxonomies should be adapted to take 
account of developments in educational research and 
instructional technology, particularlythe development 
of multimedia and internet-based instruction (Jonassen 
etal. 1999). They specifically seek to include integrated 
knowledge, ampliative (knowledge enhancement) skills, 
self-awareness and self-control in their taxonomy. 

The authors seek to combine the steps of task analysis 
and outcome classification to make them 'a concurrent 
design process’ (Jonassen, Tessmerand Hannum 
1999, 31). Theysuggestthat knowledge of a taxonomy 
can facilitate task analysis in instructional design 
(J onassen, Hannum and Tessmer 1989). Ten years later 
they make the stronger claim that 'if you are unable 
to articulate the kind of thinking (by classifying the kind 
of learning outcome required) that you expect learners 
to accomplish, you have no business trying to design 
instruction to support that learning.’ (J onassen, 
Tessmerand Hannum 1999, 31) 




The taxonomy, as shown below, has 11 broad categories 
of learning outcome, with a total of 34 sub-categories: 

■ declarative knowledge 

■ cued propositional information 

■ propositional information 

■ acquiring bodies of information 

■ structural knowledge 

■ information networking 

■ semantic mapping/ conceptual networking 

■ structural mental models 

■ cognitive component skills 

■ forming concepts 

■ reasoning from concepts 

■ using procedures 

■ applying rules 

■ applying principles 

■ complex procedures found in well-structured problems 

■ situated problem solving 

■ identifying/ defining problem space 

■ decomposing problems 

■ hypothesising solutions 

■ evaluating solutions 

■ knowledge complexes 

■ mental modelling 

■ ampliative skills 

■ generating new interpretations 

■ constructing/ applying arguments 

■ analogising 

■ inferencing 

■ self-knowledge 

■ articulating content (prior knowledge) 

■ articulating socio-cultural knowledge 

■ articulating personal strategies 

■ reflective self-knowledge 

■ articulating cognitive prejudices orweaknesses 

■ executive control strategies 

■ assessing task difficulty 

■ goal setting 

■ allocating cognitive resources 

■ assessing prior knowledge 

■ assessing progress/ error checking 

■ motivation (disposition) 

■ exerting effort 

■ persisting with task (tenacity) 

■ engaging intentionally (willingness) 

■ attitude 

■ making choices. 



Evaluation: scope and structure 

This is a broad-brush framework which can be applied 
to all kinds of learning outcome, whether cognitive, 
motororpsychosocial. It brings together cognitive, 
metacognitive, affective and conative (motivational) 
dimensions, some of which are not included in widely 
used instructional design taxonomies. While it can 
be argued that the main categories are comprehensive, 
some areas are treated in more detail than others, 
the section on motivation (dispositions) being farfrom 
complete. In a later paper, J onassen (2000) provides 
more detail about problem solving, setting out 11 types 
of problem-solving outcome along the well-structured 
to ill-structured dimension. 

The authors do not specify a single organising principle 
that they used to construct the taxonomy, although 
theydo see task analysis in terms of creating learning 
hierarchies in which a higher-order outcome (such 
as problem solving orconstructing a mental model) 
depends upon lower-order outcomes (such as concept 
formation or information processing) which need to 
be mastered bythe learner first. As it stands, however 
the taxonomy is not strictly hierarchical and is therefore 
best seen as a framework. 

We concur with the authors’ claim to have improved 
on the taxonomies of Bloom (1956), Gagne (1965, 
1985) and Merrill (1983) by including learning 
outcomes which: 

1 

reflect learned behaviours (by which they seem 
to mean abilities ratherthan skills) including 
inferencing, analogising, assessing task difficulty 
and decomposing problems 

2 

reflect cognitive structures acquired in learning 
such as structural knowledge, self-knowledge and 
mental models 

3 

are traditional, such as attitudes, procedures, rules, 
concepts and problem solving. 

In each of the main categories there are some headings 
and terms which are somewhat different from those 
used in othertaxonomies, such as 'ampliative skill' 
and 'structural knowledge' (J onassen, Beissnerand 
Yacci 1993). However, it is possible to relate these 
to otherconceptualisations. For example, a 'conceptual 
network' is what other theorists have called a 'schema' 
and 'ampliative skills’ referto aspects of critical and 
creative thinking. 

The overall structure of the framework is compatible 
with that of Marzano (2001a, 2001b) in that it 
has categories that can be fitted into Marzano’s self, 
metacognitive and cognitive sytems. 
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Evaluation: theory and values 

Gagne’s taxonomy (Gagne 1985) and Merrill’s 
component display theory (1983) are acknowledged 
bythe authors as influencing theirown thinking 
(Jonassen, Tessmerand Hannum 1999). Cognitive 
and constructivist ideas predominate and the analytic 
assumption is made that 'knowledge and human activity 
can be characterised as discrete cognitive states’ 
(Jonassen, Tessmerand Hannum 1999, 30). All of the 
learning outcomes are expressed in mentalistic rather 
than behavioural terms and some (such as 'allocating 
cognitive resources’ and 'defining problem space’) 
would be very difficult to operationalise. 

The theoretical constructs forthe framework come from 
contemporary educational research in psychology and 
sociology, as well as drawing on philosophical concepts. 
Some of the underpinning approaches are similarto 
those influencing othertaxonomists, such as Marzano 
(2001a, 2001b). Forexample, the importance given 
to metacognitive and motivational aspects of learning 
implies an active model of knowledge construction. 

The authors also acknowledge the importance of 
context for learning, by including a category of situated 
problem solving (seejonassen 1997, 2000). 

By having an 'ampliative skills’ category (by which 
they mean how a learner reasons beyond given 
information through analogyand inference), J onassen 
and Tessmer imply that the issue of transfer in learning 
is not included orwell covered in otheraccounts. 

Here they draw on Moore's work (1968) in critical and 
creative thinking and claim thatthese 'knowledge 
enhancement skills’ aim to make learning more efficient 
and personally relevant, as learners generate new 
knowledge and make meaningful connections within 
what they already know. 

Evaluation: communicability and practical 
relevance 

As the framework explicitly draws on a number of fields, 
the terminology includes some unfamiliarterms, which 
maymake it challenging forteachers. On the other 
hand, the inclusion of 'traditional' learning outcomes 
such as attitudes and concepts makes it more 
accessible. The authors have a possible communicative 
advantage in that they talk about self-knowledge and 
executive control strategies instead of metacognition 
and self-regulation. 



The framework was specifically developed to aid 
instructional designers in tasks such as mapping 
a curriculum and developing materials and 
assessments, so it is likelyto be well received by 
practitioners. J onassen and Tessmer include, for 
example, the idea of 'ill-structured' problems in their 
list, arguing that teachers and learners need to engage 
in real-world orsituated problem solving where there 
may be a range of solutions (oreven no solution 
at all). One example of the successful application 
of the taxonomy is a task analysis of what is involved 
in understanding the structure, functions and powers 
of the US Department of Defence. 

As the authors point out, the usability, 
comprehensiveness and productivity of a taxonomy 
can only be properly assessed by applying it in many 
contexts. We believe that theirframework offers 
considerable promise, especially in the field of 
computer-assisted learning for which it was designed. 
Although it has more categories than Marzano (2001a, 
2001b) uses, it is a serious contenderfor widespread 
use in post-16 as well as school-age contexts. It is 
broad in scope, focuses on a range of learning 
outcomes and can cope with the application of 
knowledge and skills in complexsituations such 
as problem solving and work-based learning. 

3.2.2 

M arzano’s new taxonomy of educational objectives 

Description and intended use 

Marzano’s initial purpose (1998) was to produce 
a theory-driven meta-analysis of educational instruction 
using categories specific and functional enough 
to provide guidance for classroom practice. In his later 
book (2001a), the theory is presented as a taxonomy 
fordesigning educational objectives, spiral curricula 
and assessments. 

As illustrated in Figure 1, the theoretical model is 
a hierarchical system in which the self system controls 
the metacognitive system which in turn controls 
the cognitive system. Each of these operates on the 
retrieved content of an individual’s knowledge domain, 
which comprises stored information and knowledge 
of mental and psychomotor procedures. This knowledge 
can be represented linguistically, non-linguistically or 
in an affective (emotional) form. 

The three systems are said to form a hierarchy in 
terms of the downward flow of information, once the 
self system has decided to engage in a task (Marzano 
2001b). Marzano (2001a) makes the additional 
claim that each level requires more conscious thought 
than the one below it. 



Table 1 

Marzano's new taxonomy 



Figure 1 

The hierarchical control 
structure of Marzano's 
theory-based taxonomy 




System 


Level 






Self 


6 


Examining the importance 
of the knowledge 
Examining efficacy 
(ability to learn) 

Examining emotions associated 
with knowledge and motivation 




Metacognitive 


5 


Specifying learning goals 
Monitoring the execution 
of knowledge 
Monitoring clarity 
Monitoring accuracy 




Cognitive 


4 


Knowledge utilisation 


Decision making 
Problem solving 
Experimental enquiry 
Investigation 




3 


Analysis 


Matching 



Classifying 
Error analysis 
Generalising 
Specifying 

2 Comprehension Synthesis 

Representation 

1 Retrieval Recall 

Execution 



At the top of the hierarchy of consciousness and control 
is the self system, in which attention and motivation are 
controlled in accordance with beliefs and calculations 
of discrepancies between perceived and desired states. 
'Because the mechanisms in the self-domain are the 
working elements that define motivation and volition 
in human behaviour they have historically been referred 
to as conative structures’ (2001a, 10). The self system 
is said to exert control over the metacognitive system, 
which is concerned with goal specification, process 
specification, process monitoring and disposition 
monitoring. The metacognitive system in turn 'exerts 
control overthe cognitive system that operates in the 
knowledge domains’ (2001a, 65). The functions of 
Marzano's three systems are shown in Table 1, grouped 
into six levels. 



Marzano (2001b) sees Levels 1-4 within the cognitive 
system as hierarchical, in that knowledge retrieval 
is a prerequisite for comprehension, which is 
a prerequisite foranalysis, without which knowledge 
cannot be used. 

Foreach of the functions atthe six levels of his 
taxonomy, Marzano (2001b) provides forteachers 
illustrative instructional objectives, cues or questions. 
These, however, add little further meaning, as the 
question forgeneralising shows: 'What generalisations 
can be inferred from this knowledge?’ (2001b, 187). 

The knowledge domain is comprised of declarative 
and procedural knowledge. Declarative knowledge 
is subdivided into organising ideas (principles and 
generalisations) and details (episodes, cause-effect 
sequences, time sequences, facts and vocabulary 
terms). Procedural knowledge is said to consist of more 
or less complex mental and psychomotor processes 
and skills. Mental skills are broken down into tactics, 
algorithms and single rules. 
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Evaluation: scope and structure 

The scope of Marzano’s taxonomy is certainly very 
broad, based as it is on a theory of thinking and learning 
which aims to be comprehensive. It covers objectives 
which relate to mental activity, values, beliefs and 
dispositions as well as observed behaviour. It builds 
on his earlier work (eg Marzano etal. 1988; Marzano 
1992), but differs in that it has relatively little to say 
about creative thinking. It takes account of conative 
and affective aspects of thinking, but does not attempt 
to account forindividual and situational differences 
in those domains. 

The knowledge categories proposed by Marzano 
appearto be comprehensive, not least because they 
are said to contain all 'facts' and 'vocabularyterms'. 
However, he does not include 'pattern' or 'system' 
in his list, norterms which referto probabilistic 
knowledge of social situations. Marzano’s treatment 
of sub-categories is generally in need of greater 
justification: forexample, it is not self-evident that the 
sub-categories of knowledge utilisation are mutually 
exclusive northat they offer comprehensive coverage. 

The taxonomy has not been tested to see whether 
its structure is sufficiently robust to ensure consistent 
classification of instructional objectives and/orthinking 
skills. One possible area of confusion is the inclusion 
of error analysis in the cognitive ratherthan the 
metacognitive system. Marzano uses this heading 
to coverthe evaluation of the logic and reasonableness 
of knowledge claims and provides a list of informal 
fallacies which can be detected through critical thinking. 
However, while the evaluation of another person’s 
thinking is a cognitive activity, monitoring and detecting 
errors in one’s own thinking involves metacognition. 

Marzano (2001a, 2001b) makes many comparisons 
between his new taxonomy and Bloom’s taxonomy 
of educational objectives (cognitive domain) 

(1956). The main differences are his addition of the 
metacognitive and self systems and his replacement 
of complexity with flow of information as an organising 
principle. Other differences lie largely in the detail, 
but especially in Marzano’s treatment of analysis 
which incorporates elements from Bloom’s higher-order 
categories of analysis, synthesis and evaluation. 



The three-tier structure of Marzano’s taxonomy has only 
a modest level of empirical support from his own highly 
ambitious meta-analysis of research on instruction 
(1998). The mean differences in achievement gain 
produced by educational interventions making use 
of the three systems are not great (27 percentile points 
forthe self system, 26 forthe metacognitive system 
and 21 forthe cognitive system). The standard 
deviations are so large that it is simply not possible 
to argue that it is better to aim forchange via the 
self system ratherthan through the cognitive system. 
Conative, affective and cognitive aspects of thought 
are not easily separable and all three are involved 
in planning, monitoring and evaluating. Marzano 
seems to think of the metacognitive system as a sort 
of computer unlike the self system which deals with 
motivation, beliefs and feelings. This does not accord 
with subjective experience, in which motivation, beliefs 
and feelings are not disassociated from planning, 
monitoring and evaluating, either when engaging in 
an activity or when seeing itthrough. 

The flow of information is more complex and interactive 
than Marzano suggests. It cannot all be downwards, 
as Marzano claims. There are manyexamples in 
the literature of self-concept and self-efficacy being 
enhanced as a result of cognitive skill acquisition. 

The brain does not function like a strictly hierarchical 
military organisation. Representations of the self 
are formed through experiences at all levels of 
consciousness and control. In Demetriou’s empirically 
supported model of the mind, there is a great deal 
of dynamic interaction between levels and modules 
(Demetriou and Kazi 2001). 

Marzano’s claims about the flow of information, the 
amount of conscious thought needed at each level and 
the dependence of each level on those below are largely 
open for experimental enquiry. When he states that 
without a cleargoal, task execution will break down 
(2001b), he is evidently not thinking about many stages 
of creative thinking; and when he states thatdecision 
making requires analysis, he is ignoring the widespread 
use of heuristics that reduce mental effort. 

Evaluation: theory and values 

Marzano began by constructing a theory to support his 
taxonomic framework. He does not discuss the values 
implicit orexplicit in his theory, but it is certainlynot 
value-neutral, as it is highly individualistic, using the 
metaphor of a control system with a powerful authority 
in charge. At the same time, the taxonomy seems 
designed to support the authoritative role of the teacher 
who introduces knowledge objectives and tries to 
enthuse learners into adopting them as theirown. 

The hierarchical control feature of the taxonomy seems 
less compatible with a participatory, enquiry-based 
approach to learning. 



In Marzano’s writing, there is an all-pervasive emphasis 
on rationality, to such a degree that emotions seem 
to be there not to be experienced, but to be analysed 
to see if theyare reasonable. At the same time, he 
is a pragmatist, suggesting that teachers should base 
their practice on the research evidence of 'what works’, 
as established through meta-analysis. 

So faras the school curriculum is concerned, Marzano 
argues againstsimplyusing information load as an 
index of progression, suggesting that his levels should 
be built into a spiral curriculum that emphasises 
process more than content. 

We have here a largely coherent theory which draws 
on a wide range of research in cognitive and educational 
psychology, ratherthan on theory development in 
the fields of critical and creative thinking. Its basic 
three-tier structure is similarto that developed 
independently by Demetriou and Kazi (2001). Both 
Marzano and Demetriou distinguish between cognitive, 
metacognitive (termed hypercognitive by Demetriou) 
and self systems (self-representation for Demetriou). 
Demetriou and Kazi have accumulated an impressive 
amount of empirical supportfortheirmodel, which 
is even broaderin scope than Marzano’s. Unlike 
Demetriou and Kazi, Marzano pays little attention 
to non-cognitive aspects of personality and says little 
about sensitivities to the situational and interpersonal 
factors which affect learning. It remains to be seen 
howfarhis theory will yield verifiable predictions 
and findings with practical implications forteachers 
and learners. 

Although he provides cleardefinitions and examples, 
there are some instances in which Marzano defines 
terms in unfamiliar ways. Forexample, he defines 
synthesis as 'the process of distilling knowledge down 
to its key characteristics’ (2001a, 34), which contrasts 
with the more familiar idea of putting together parts so 
as to form a (sometimes complex) whole. For Marzano, 
matching involves the detection of differences as well 
as similarities and in this case the term 'compare' 
or 'compare and contrast’ would be more appropriate. 

Evaluation: communicability and practical 
relevance 

Marzano’s theory has considerable potential for use 
in instructional design, teaching, assessment, research 
and evaluation. Fie summarises both general and 
specific instructional implications for practitioners, 
manyof which are applicable irrespective of the 
age range of learners. His inclusion of a knowledge 
utilisation level, dealing with the orchestration of 
thinking, makes the taxonomy meaningful in real-life 
problem-solving contexts and this feature, together 
with the importance given to the metacognitive and 
self systems could help to bring about improvements 
in formative and summative assessment of the 
kind that is already being developed in relation to 
higher-level key skills objectives. 



The important features of Marzano’s new taxonomy 
are as follows. 

■ It provides a coherent framework for classifying 
instructional goals, though some category boundaries 
are unclearand there are some weak areas, such as 
the affective aspects of learning. 

■ It promotes the use of clearstatements of 
educational goals. 

■ It is applicable in all contexts of teaching and learning. 

■ It is not too complex for everyday use. 

■ It may have the effect of limiting creativity. 

■ It may not support collaborative learning effectively 
because of its individual focus. 

■ It is likelyto prove meaningful and useful to teachers 
and othereducational professionals, to stimulate 
various forms of enquiry, and to help teachers to 
systematise and improve their practice. 

3.3 

Evaluation of instructional design frameworks 

Here we describe and evaluate two very different 
frameworks, although both sets of authors are 
concerned about the complexity of thought. The SOLO 
taxonomy of Biggs and Collis (1982) is primarilyan 
assessment tool, whereas Anderson and Krathwohl’s 
(2001) revision of Bloom's taxonomy is broaderin 
scope. In Appendix 2, we present six otherframeworks, 
including the ground-breaking work of Bloom and 
his team in 1956. We include Feuerstein’s framework 
(1980), which illustrates his distinctive Instrumental 
Enrichment (IE) approach to developing thinking and 
learning. Williams (1970) is one of the few authors 
to concentrate on creativity. 

3.3.1 

Biggs and Collis's SOLO taxonomy: 
structure of the observed learning outcome 

Description and intended use 

The taxonomy was devised by Biggs and Collis 
(1982; Biggs 1995, 1999). According to the authors, 
it provides a systematic wayof describing how 
a learner’s performance grows in complexity when 
mastering many tasks, particularlythe sort of tasks 
undertaken in schools and colleges. Theyargue 
fora general sequence in the growth of the structural 
complexity of many concepts and skills; that sequence 
maythen be used to identify specific targets orto help 
teachers to assess particular outcomes. It therefore 
attempts to describe the level of increasing complexity 
in a student’s understanding of a subject through five 
stages, and it is claimed to be applicable to any subject 
area. Not all students get through all five stages, and 
not all teaching (and even less 'training') is designed 
to take them all the way. The stages are described 
in outline below and with additional criteria in Table 2. 
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Table 2 

The SOLO taxonomy 
levels, with descriptors 
and criteria 



SOLO description 


Capacity 


Relating operation 


Consistency and closure 


Pre-structural 


Minimal: cue and response confused 


Denial, tautology, transduction 
Bound to specifics 


No need felt for consistency: 
closure without seeing the problem 


Uni-structural 


Low: cue and one relevant datum 


Can generalise only in terms of one 
aspect 


No need felt for consistency: 
closed too quickly; jumps to 
conclusions, so can be very inconsistent 


M ulti-structural 


Medium: cue and isolated relevant data 


Can generalise only in terms of a few 
limited and independent aspects 


Feeling for consistency: closure too 
soon on basis of isolated fixations so 
can reach different conclusions with 
same data 


Relational 


High: cue and relevant data and 
interrelations 


Induction: can generalise within 
given or experienced context using 
related aspects 


No inconsistency in given system, but 
closure is unique to given system 


Extended abstract 


Maximal: cue and relevant data and 
interrelations and hypotheses 


Deduction and induction: can generalise 
to situations not experienced 


Inconsistencies resolved: no need for 
closed decisions; conclusions held open 
or qualified to allow logically possible 
alternatives 



1 

Pre-structural: here students are simply acquiring bits 
of unconnected information which have no organisation 
and make no sense. 

2 

Uni-structural: simple and obvious connections are 
made, buttheirsignificance is not grasped. 

3 

Multi-structural: a number of connections maybe made, 
but the meta-connections between them are missed, 
as is theirsignificance forthe whole. 

4 

Relational: the student is now able to appreciate the 
significance of the parts in relation to the whole. 

5 

Extended abstract: the student makes connections 
not only within the given subject area, but also beyond 
it, and is able to generalise and transferthe principles 
and ideas underlying the specific instance. 



The taxonomy (see Table 2 above) explicitly draws 
on a Piagetian framework, but differs from Piaget's 
(1952) classical stage theory in that Biggs and Collis 
argue that each stage does not so much replace 
the previous stage as add to the repertoire of available 
cognitive responses. In different situations, learners 
may'regress’ to an earlier mode of functioning 
or use a higher cognitive function in the learning 
of a lower-order one, adopting a 'multi-modal' 
approach to the task at hand (Biggs and Collis 1991). 
The taxonomy is based on an analysis of the work of 
several hundred pupils of different ages across a range 
of subjects and the identification of recurring patterns 
in pupils’ thinking. Biggs and Collis found a general 
age-related progression through secondary schooling, 
from multi-structural to relational to extended abstract 
thinking (equivalent to Piaget's formal operations stage 
and not usually reached before the age of 16). 

The purpose of the SOLO taxonomy is to provide 
a systematic way of describing how a learner's 
performance grows in structural complexity when 
tackling and mastering a range of tasks. It can 
therefore be used to identifyand define curriculum 
objectives that describe performance goals ortargets, 
as well as forevaluating learning outcomes, so that 
the levels at which individual students are performing 
oroperating can be identified. 



Evaluation: scope and structure 

The SOLO taxonomy can be used to classify the 
quality of students’ responses to assessment items, 
at least as represented by identifying the sophistication 
of the assumed underlying logic. It has been used 
experimentally in a wide range of studies; for example, 
to evaluate the learning associated with the use 
of the computerlanguage LOGO (Hawkins and Hedberg 
1986) as well as in the context of higher education 
(Boulton-Lewis 1995). The keyterms and definitions 
have remained the same forover20 years, as it has 
been developed and applied. It offers a framework 
forthe assessment of challenging aspects of learning 
such as the understanding of concepts, and forproblem 
solving (Collis and Romberg 1991: foran overview 
and summary of research, see Prosserand Trigwell 
1999). The taxonomy has also proved effective as 
a means of planning and developing curricula based 
on the cognitive characteristics of the learners. 

Evaluation: theory and values 

The SOLO taxonomy identifies a developmental 
progression of a student's cognitive responses, 
based on Piaget's theoryof genetic epistemology. 

It is therefore open to some of the same criticisms that 
have been levelled against stage theories (in terms 
of the relationship between, and progression through, 
the stages) and makes some implicit assumptions 
aboutthe nature of knowledge. The taxonomydoes 
not take into account the social nature of interactions 
orthe influence of affective and conative dimensions 
of thinking, because its focus is on students’ 
performance (in a particularcontext at a particular 
time). It therefore assumes that the tasks used are 
understood by students and offer effective contexts 
forsuch assessment. 

There are links between the approach of Biggs and 
Collis and other work on learning, such as Saljo’s 
conceptions of learning (1979a, 1979b), and Bateson’s 
levels of learning (1973). 

Evaluation: communicability and practical 
relevance 

The wide and effective application of the SOLO 
taxonomy by educational researchers, curriculum 
designers and teachers at all levels of education and 
across a wide range of subjects indicates its practical 
value and the ease with which it can be understood. 

It was designed to improve the quality of learning and 
the quality of feedback given to students, orfor use 
by students in self-assessment. The five levels are easy 
to grasp, although the qualitative nature of judgements 
needed to assign observed outcomes to the appropriate 
categories or levels of the taxonomycan make 
it challenging to applythe taxonomy consistently. 

Although the formative value of the SOLO taxonomy 
is apparent, there is some risk that it could be used 
to devise curricula for novice learners in the belief 
thatthey are not ready to build connections between 
related ideas. 



3.3.2 

Anderson and Krathwohl's revision of Bloom's 
taxonomy of educational objectives 

Description and intended use 

This revision of Bloom's framework for categorising 
educational objectives was undertaken to refocus 
attention on Bloom's taxonomy (1956) and to 
incorporate the many advances in knowledge since 
the original publication. The revision took account 
of international feedback and Bloom, together 
with several of his co-authors, contributed to several 
chapters in this work. 

The original framework comprised the following six 
categories: knowledge, comprehension, application, 
analysis, synthesis and evaluation. With the exception 
of knowledge, all categories were labelled as 'abilities 
and skills’; and foreach of these, knowledge was 
deemed a prerequisite. The categories were presumed 
to constitute a cumulative hierarchy; that is, each 
category was conceived as building on and comprising 
a more advanced achievement than its predecessor. 

The Anderson and Krathwohl revision (2001) retains six 
cognitive process categories: remember, understand, 
apply, analyse, evaluate and create. These correspond 
closely to the Bloom categories, and since the revision 
draws heavily on Bloom, it is worth identifying the 
changes incorporated into the revision. 

Changes in emphasis 

The revision emphasises the use of the taxonomy in 
course planning, instruction and assessment; and 
in aligning these three. The authors view this as a major 
shift from the original handbook, where the focus was 
on providing extensive examples of test items in each 
of the six categories. Other significant changes are 
listed below. 

■ While the original handbook was developed by 
college examiners, the revision is designed to be 
of use by elementary and high-school teachers. 

■ Sample assessmenttasks contained within the revision 
are designed to illustrate and clarifythe meaning of the 
various sub-categories. Theyare not included as model 
test items, as in the original handbook. 

■ The original handbook made use of test items to 
clarifythe meaning of definitions; in the revision, 
meanings are clarified through extensive descriptions 
of sub-categories and case vignette illustrations. 

■ It is no longer claimed that the process categories form 
a cumulative hierarchy where the learnercannot move 
to a higher level without mastering all those below it. 
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Figure 2 

Structural changes from 
Bloom to the Anderson 
and Krathwohl revision 



Separate dimension 
(knowledge) 




Factual 

Conceptual 

Procedural 

Metacognitive 



Knowledge 


(in verb form) 


Remember 




Comprehension 


Understand 




Application 


Apply 




Analysis 


Analyse 




Synthesis 


Evaluate 


Evaluation 




Create 



Cognitive process dimension 



Original framework 



Revised framework 



Changes in terminology 

■ Educational objectives indicate that a student should 
be able to do something (verb) to or with something 
(noun). In the original framework, nouns were used 

to describe the knowledge categories (eg application). 

In the revision, the majorcategories in the cognitive 
process dimension have been relabelled with verb 
forms (eg apply). Knowledge as a cognitive process 
is renamed remember. Sub-categories in the cognitive 
process dimension have also been labelled with 
verbs, such as checking and critiquing (sub-categories 
of evaluate). 

■ The revision has renamed and reorganised the 
knowledge sub-categories as fourtypes of knowledge: 
factual, conceptual, procedural and metacognitive. 

■ Two of the majorcategories in the original framework 
have been renamed: comprehension has become 
understand and synthesis has become create. 

Changes in structure 

Anderson and Krathwohl's taxonomy (2001) involves 
a two-dimensional table, with six cognitive processes 
and fourtypes of knowledge. Figure 2 summarises 
the structural changes from Bloom’s original framework; 
the examples of learning objectives, activities 
and assessment shown in Figure 3 illustrate why 
it is useful to separate the 'knowledge' and 'cognitive 
process’ dimensions. 



The revised framework orders the six cognitive process 
categories according to theirdegree of complexity. 

In the original framework, it was claimed that mastery 
of a more complex category required mastery of all 
the preceding, less complex categories. Anderson and 
Krathwohl state that empirical evidence only supports 
a cumulative hierarchy for Bloom’s middle three 
categories of comprehension, application and analysis. 
Flowever, they confirm that the revised framework 
remains hierarchical in overall complexity. 

Throughout the book, the authors use four organising 
questions to show how the taxonomy framework can be 
used to support teachers in the classroom. 

■ The learning question: what is important for students 
to learn in the limited school and classroom time 
available? 

■ The instruction question: how does one plan and deliver 
instruction that will result in high levels of learning for 
large numbers of students? 

■ The assessment question: how does one select or 
design assessment instruments and procedures that 
provide accurate information about how well students 
are learning? 

■ The alignment question: how does one ensure that 
objectives, instruction and assessment are consistent 
with one another? 





Figure 3 Cognitive process dimension 



Taxonomy table with 
illustrative examples 


1 

Remember 


2 

Understand 


3 

Apply 


4 

Analyse 


5 

Evaluate 


6 

Create 




1 

Recognising 

2 

Recalling 


1 

Interpreting 

2 

Exemplifying 

3 

Classifying 

4 

Summarising 

5 

Inferring 

6 

Comparing 

7 

Explaining 


1 

Executing 

2 

Implementing 


1 

Differentiating 

2 

Organising 

3 

Attributing 


1 

Checking 

2 

Critiquing 


1 

Generating 

2 

Planning 

3 

Producing 


Knowledge dimension 














■ Factual knowledge 

■ knowledge of terminology 

■ knowledge of specific 
details and elements 


Example 
assessment 
Quiz on 
addition facts 










Example activity 
Rewrite a scene 
from Macbeth 
in a modern idiom 


■ Conceptual knowledge 

■ knowledge of 
classifications and 
categories 

■ knowledge of principles 
and generalisations 

■ knowledge of theories, 
models and structures 




Example 

learning objective 
Understand the 
theory of plate 
tectonics as an 
explanation for 
volcanoes 




Example 

learning objective 
Select sources 
of information 
related to 
writing about 
a historical figure 


Example 

learning objective 
Evaluate food 
commercials from 
a set of principles 


Example activity 
Rewrite a scene 
from Macbeth 
in a modern idiom 


■ Procedural knowledge 

■ knowledge of subject- 
specific skills and 
algorithms 

■ knowledge of subject- 
specific techniques and 
methods 

■ knowledge of criteria for 
determining when to use 
appropriate procedures 






Example 

learning objective 
Gain a working 
knowledge of 
memorisation 
strategies 






Example 

learning objective 
Create a 
commercial 
that reflects 
understanding of 
how commercials 
are designed to 
influence people 


■ Metacognitive 
knowledge 

■ strategic knowledge 

■ knowledge about 
cognitive tasks, including 
appropriate contextual 
and conditional 
knowledge 

■ self-knowledge 




Example 

learning objective 
Understand 
the efficiency 
of memorisation 
strategies 
(in certain 
circumstances) 






Example 

learning objective 
Check the 
influences of 
commercials on 
students' 'senses’ 
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A series of vignettes based on actual classroom 
practice is used to demonstrate how the taxonomy table 
can be used to aid understanding of the complex nature 
of classroom instruction. It is claimed that increased 
understanding of the framework can result in improving 
the quality of classroom instruction, not least by 
encouraging teachers to include more complex cognitive 
process categories in classroom instruction. First and 
foremost, the taxonomy table should be used as an 
analytical tool to enable teachers to conduct a deeper 
examination of the alignment of learning objectives, 
instruction and assessment. 

Evaluation: scope and structure 

While acknowledging that almost every cognitive 
objective involves an affective component, the authors 
judged that inclusion of the affective domain would 
create an overly complex taxonomy, which would not, 
forthat reason, become widely adopted. However; the 
authors considerthattheirrevised cognitive domain 
taxonomy 'does contain some seeds for future affective 
development' in that metacognitive knowledge goes 
some way to bridging the cognitive and affective 
domains (2001, 301). Mayer (2002) argues that in 
addition to the new category of metacognition, the 
revised taxonomy recognises the role of metacognitive 
and motivational processes in that it clarifies their 
role within the cognitive process dimension and in 
particular, within the categories of create and evaluate. 

The widely used terms 'critical thinking’ and 'problem 
solving’ have not been included as majorcategories 
within the taxonomy, since the authors view these 
terms as having similarcharacteristics to theircategory 
understand. However, they maintain that, unlike 
understand, critical thinking and problem solving tend 
to involve cognitive processes in several categories 
across theircognitive process dimension. 

The creation of a matrix whereby cognitive processes 
operate with different types of subject matter content 
(ie knowledge) provides teachers with a useful tool 
to help them to analyse theirteaching objectives, 
activities and assessment. Classifying learning 
objectives within the framework is likely to increase 
a teacher’s understanding of each objective and help 
them to plan ways to ensure that pupils succeed. 
Classifying longer units of work allows teachers to make 
choices relating to coverage across both dimensions. 

We note, also, that this approach has been taken by 
several othertheorists: Romiszowski (1981), J onassen 
andTessmer(1996/97), forexample. 



There is little to choose between Bloom (1956) and 
Anderson and Krathwohl (2001) in theirtreatment 
of three sub-categories of knowledge. The revised term 
for Bloom’s knowledge of specifics is factual knowledge. 
What Bloom calls knowledge of ways and means is now 
called procedural knowledge, and Bloom’s knowledge 
of the universal and abstractions in a field is now 
labelled as conceptual knowledge. However, while 
Bloom made implicit references to what we now call 
metacognition, Anderson and Krathwohl explicitly 
list 'metacognition’ as a type of knowledge. But it 
is open to question whetherthe term 'metacognition' 
refers to knowledge of a different type. It is not uniquely 
distinguished bythe processes involved (knowing that, 
knowing how and understanding ideas), but rather 
by its content. 

While Anderson and Krathwohl give weight to the 
separate classification of metacognitive knowledge, 
theydo not explicitly address the monitoring, control 
and regulation of students' cognition, arguing that 
this involves 'different types of cognitive processes 
and therefore fits into the cognitive process dimension’ 
(2001, 43). When addressing metacognition within the 
knowledge dimension, the authors provide a rationale 
forthe inclusion of metacognitive knowledge - 
a comprehensive overview of each of the three types 
of metacognitive knowledge together with illustrative 
examples. Theirtreatment of metacognition within 
the cognitive process dimension attracts little attention 
and provides the readerwith onlytwo examples. This 
decision results in an inconsistent treatment of the 
two aspects of metacognition (knowledge and 
self-regulation). 

Evaluation: theory and values 

Theoretical advances in educational psychology 
and, to a lesser extent, in cognitive psychology have 
contributed to this revision of Bloom's framework. 

The focus on knowledge types and the delineation of 
process categories into specific cognitive processes is 
based largelyon 'an examination of otherclassification 
systems’ (2001, 66), dating from 1969 to 1998, and 
including Sternberg's model (1998) of 'successful 
intelligence'. While acknowledging that the framework 
should ideally be based on a single, widely accepted, 
and functional theory of learning, the authors note that, 
despite recent advances, we are still without a single 
psychological theorythatadequatelyprovides a basis 
for all learning. The framework reflects the authors’ 
belief that knowledge is structured bythe learnerin 
line with a rationalist-constructivist tradition. They 
do not adhere to the idea that knowledge is organised 
in stages or in system-wide logical structures, as in 
traditional 'developmental stage' models of thinking. 

Anderson and Krathwohl claim thattheirtaxonomy 
is 'value neutral and therefore can be used by those 
operating from a variety of philosophical positions’ 
(2001, 296). This is broadlytrue, despite the 
implication (equally present in Bloom's taxonomy) that 
more complex thinking is usually more highly valued. 



Evaluation: communicability and practical 
relevance 

The terminology is clearand accessible, and teachers 
alreadyfamiliar with the original taxonomy will find 
little difficulty in assimilating the revised structure 
and terminology. The inclusion of a series of vignettes 
helps to illustrate the key concepts and elements in 
the taxonomytable. Although these examples are taken 
from school contexts, the taxonomy is also suitable for 
post-16 application, as was Bloom’s original framework. 

The taxonomy was designed to help teachers 
to understand and implement a standards-based 
curriculum. The authors expect the framework to 
be used mainly by teachers who are given a set 
of objectives and are expected to deliver instruction 
that enables a large proportion of pupils to achieve 
the expected standard. The dominant theme running 
throughout the text is the alignment of learning 
objectives, instruction and assessment. The taxonomy 
encourages teachers to focus on coverage, thereby 
allowing students to experience learning opportunities 
across the cognitive domain. The purpose of 
the framework is to help teachers to clarify and 
communicate what they intend theirstudents to learn. 
The authors are less concerned with how teachers 
teach, since it is their view that most instructional 
decisions depend on the teacher’s creativity, ingenuity 
and wisdom. 

There are several reasons why the taxonomy may prove 
attractive to practitioners. It does not seek to change 
radically howtheyteach orto challenge their beliefs 
aboutteaching and learning. The authors use language 
thatteachers are familiar with, and exemplify use of 
the taxonomy with detailed case studies that reflect 
current classroom practice. 

This revision of Bloom's taxonomy- with its emphasis 
on helping teachers to align learning objectives, 
instruction and assessment - is a strong contender 
for use in post-16 education. 

3.4 

Evaluation of critical/productive thinking 
frameworks 

Here we present for comparison two highly influential 
critical thinking frameworks (those of Ennis and Paul) 
and one that is more descriptive (by Halpern). Ennis 
and Halpern both include creative thinking skills 
in their treatment, butto a lesserextentthan Lipman 
(see Appendix 2). Also in Appendix 2, J ewell (1996) 
accepts Lipman’s 'three C’s’ framework of critical, 
creative and caring thinking, while Allen, Feezel and 
Kauffie (1967) are concerned only with argument 
analysis and informal logic. We also evaluate two 
overviews, those of Baron (1985) and Gubbins (1986). 



3.4.1 

Ennis's taxonomy of critical thinking dispositions 
and abilities 

Description and intended use 

Ennis’s views have developed overtime (he has been 
publishing in this area since 1962) and there have 
been significant changes in his thinking, particularly 
in the area of critical thinking dispositions (eg 1996). 
However his basic definition has remained constant, 
worded as follows: 'Critical thinking is reasonable 
and reflective thinking that is focused on deciding what 
to believe ordo’ (1985, 45). His intention is to provide 
a rationale forthe teaching of critical thinking and 
a taxonomy of 'goals forcritical thinking’ (1985, 46) 
oran 'outline of a conception of critical thinking’ 

(1998, 17). He claims thatthe significant features 
of this taxonomyare as follows. 

■ It focuses on belief and action. 

■ It contains statements in terms of things that people 
actually do orshould do. 

■ It includes criteria to help evaluate results. 

■ It includes both dispositions and abilities. 

■ It is organised in such a waythat it can form the 
basis fora thinking-across-the-curriculum programme 
as well as a separate critical thinking course atthe 
college level. 

Although Ennis includes creative thinking in this 
definition, he considers that critical thinking is not 
equivalentto higher-order thinking, since critical 
thinking also involves dispositions. He proposes 
a set of six criteria forjudging a set of critical thinking 
dispositions: simplicity, comprehensiveness, 
value, comprehensibility, conformity of its language 
to our everyday meanings and the fitting of subordinates 
(if any) under superordinates. He rejects a further 
criterion, mutual exclusivity, on the basis of 
comprehensibility (1996). He claims that in order to 
ensure that categories in a critical thinking taxonomy 
do not overlap, it becomes necessary to redefine 
words with such precision thattheycan no longer 
be easily understood. 

The 1998 version of his taxonomy, which is 
summarised below, consists of three main dispositions 
(with sub-categories) and 15 abilities presented as 
a list (some with sub-categories) to provide a 'content 
outline’ fora critical thinking curriculum. The original 
1987 version contained a longerand more complex list 
of abilities and sub-categories. Ennis does notclaim 
that either list is exhaustive. 
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Dispositions 

Critical thinkers: 

1 

Care that their beliefs are true, and that their 
decisions are justified; that is care to 'get it right' 
to the extent possible, or at least care to do the best 
they can. This includes the interrelated dispositions 
to do the following: 

■ seek alternatives (hypotheses, explanations, 
conclusions, plans, sources) and be open to them 

■ endorse a position to the extent that, but only to the 
extentthat, it is justified by the information available 

■ be well informed 

■ seriouslyconsiderpoints of view other than theirown. 

2 

Representa position honestlyand clearly (theirown 
as well as others’). This includes the dispositions to 
do the following: 

■ be clear a bout the intended meaning of what is said, 
written, or otherwise communicated, seeking as much 
precision as the situation requires 

■ determine and maintain focus on the conclusion 
orquestion 

■ seek and offer reasons 

■ take into account the total situation 

■ be reflectively aware of theirown basic beliefs. 

3 

Care about the dignity and worth of every person. 

This includes the disposition to: 

■ discoverand listen to others’ views and reasons 

■ take into account others’ feelings and level of 
understanding, avoiding intimidating orconfusing 
others with their critical thinking prowess 

■ be concerned about others' welfare. 

Abilities 

Ideal critical thinkers have the abilityto: 

clarify 

1 

identifythe focus: the issue, question, orconclusion 

2 

analyse arguments 

3 

ask and answerquestions of clarification 
and/ orchallenge 

4 

define terms and judge definitions and deal with 
equivocation. 



judge the basis for a decision 

5 

judging the credibility of a source 

6 

observe and judge observation reports 

infer 

7 

identify unstated assumptions 

8 

deduce and judge deductions 

9 

induce and judge inductions 

■ to generalisations 

■ to explanatory conclusions 

10 

make and judge value judgements. 

make suppositions and integrate abilities 
11 

considerand reason without letting the 
disagreement or doubt interfere with theirthinking 
(suppositional thinking) 

12 

integrate the otherabilities and dispositions in making 
and defending a decision. 

use auxiliary critical thinking abilities 

13 

proceed in an orderly mannerappropriate to the 
situation; for example, 

■ follow up problem-solving steps 

■ monitortheirown thinking 

■ employa reasonable critical thinking checklist 

14 

be sensitive to the feelings, level of knowledge, 
and degree of sophistication of others 

15 

employ appropriate rhetorical strategies in discussion 
and presentation. 



Evaluation: scope and structure 

Ennis defines the basic areas of critical thinking 
as clarity, basis, inference and interaction, which 
he has then broken down into the listof abilities. 

He acknowledges the importance of the content domain 
in which critical thinking is applied. He acknowledges 
that his taxonomy does not incorporate suggestions 
for 'level, sequence and repetition in greaterdepth, 
emphasis orinfusion in subject matterarea, which might 
be eitherexclusive or overlapping’. He claims thatthe 
first two dispositions are 'essential' forcritical thinking 
and that the third, sensitivity to others, is 'correlative' 
and desirable ratherthan 'constitutive' (1996, 171). 

The 'taxonomy' is therefore a list of dispositions 
and abilities relevantto critical thinking. Ennis does 
not include reflection as a major heading, despite its 
explicit role in his definition of critical thinking. 

Evaluation: theory and values 

The underpinning values of Ennis’s work are those 
of rationality and logical thinking, with little attention 
paid to the impact of feelings on thinking. Although 
he acknowledges the importance of recognising and 
valuing others, application of his taxonomy requires 
a level of detachment. Ennis has been challenged 
by M a rtin (1992) on the potential creation of such 
a 'dangerous distance' required forcritical thinking 
which does not accommodate affective motives 
and reasons. Ennis defends critical thinking against 
cultural bias (1998), though accepting that culture 
and context have serious implications forsuch 
an approach because of the difficulty of balancing 
competing cultural perspectives. He has also 
vigorously defended the concept of critical thinking 
dispositions against subject specificity [eg McPeck’s 
(1992) concerns] and considers this objection from 
empirical, epistemological and logical perspectives. 

Evaluation: communicability and practical 
relevance 

Ennis aimed to produce a taxonomy which enables 
critical thinking to be used practically. He says that his 
taxonomy is 'simple and comprehensible' (1996, 173); 
and states that he considers that it can be implemented 
successfully in different ways, though he acknowledges 
that it needs further research to validate detailed 
aspects. As itstands, itshould be particularly useful 
foranalysing curriculum units in critical thinking orfor 
auditing subject-specific critical thinking programmes. 
However, the numberand relevance of the broad 
categories and theirsub-categories to particular fields 
may make it somewhat daunting to apply. Its strength 
is its identification of a helpful framework for use in 
formulating particulareducational goals, or, as Ennis 
states: 'In engaging in such thinking, one is helped by 
the employment of a set of critical thinking dispositions 
and abilities' (1998, 17). 



Finally, as Ennis himself argues, the assessment 
of critical thinking is problematic. He analyses different 
approaches to assessing critical thinking, rejecting 
multiple-choice assessment for all but self-assessment 
and research. He also questions performance-based 
assessment on grounds of cost, focus and context 
(the more realistic the performance, the more complex 
the problem). Context-based assessments require 
information gathered overtime and across a range 
of situations (Blatz 1992). 

3.4.2 

Halpern's reviews of critical thinking skills and 
dispositions 

Description and intended use 

Much of the material presented here first appeared 
in Halpern's influential book on critical thinking 
(1984), and was laterdeveloped into 'a taxonomy 
of critical thinking skills’ (1994, 31). The taxonomy 
was intended to provide a basis forthe national 
assessment of critical thinking skills in adults in the 
US. At a government-sponsored workshop held in 1992, 
Halpern referred to the thinking skills 'needed to 
compete in a global economyand in the exercise 
of citizenship’ (1994, 29), but chose to focus on what 
is often referred to as 'higher-order thinking' - 'thinking 
that is reflective, sensitive to context and monitored’. 
She used the following category headings: 

■ verbal reasoning skills 

■ argument analysis skills 

■ skills in thinking as hypothesis testing 

■ using likelihood and uncertainty 

■ decision-making and problem-solving skills. 

The 1992 workshop was set up in response to the 
following US national objective (National Education 
Goals Panel 1991, 237): 'The proportion of college 
graduates who demonstrate an advanced ability 
to think critically, communicate effectively, and solve 
problems will increase substantially.’ 

However, the workshop participants failed to agree 
on a single theoretical framework on which to base 
the proposed national assessment and the idea was 
eventually abandoned. Halpern subsequently revised 
her lists and presented them not as a taxonomy, but as 
a set of chapter reviews in her book Critical thinking 
across the curriculum (1997). This book, which closely 
follows the chapter structure of her 1984 volume on 
critical thinking, includes material on memory skills and 
on creative thinking as well as on the types of thinking 
included in her 1992 taxonomy. 
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Halpern (1997, 4) employs the following working 
definition of critical thinking as 'the use of cognitive 
skills or strategies that increase the probability 
of a desirable outcome ... thinkingthat is purposeful, 
reasoned, and goal-directed ... and effective for 
the particular context and type of thinking task’. This 
definition is so broad that it covers almost all thinking 
except basic arithmetical calculation and other 
automatised procedures. Halpern justifies herinclusion 
of memory skills by claiming (1997, 19) that 'All thinking 
skills are inextricably tied to the abilityto remember.' 

All the thinking skills described by Halpern in 
separate chapters of her book are listed in Table 4 
(see page 30), together with some category descriptors 
from Halpern (1994). What is omitted are more detailed 
descriptions and examples of use, all of which were 
written fora general readership and for'any course 
where critical thinking is valued' (1997, vii). Table 3 
illustrates the level of detail provided throughout, 
using a single example taken from Halpern’s review 
of decision-making skills. 

As Halpern’s overriding purpose is to have her readers 
use critical thinking skills, she provides a general- 
purpose framework to guide the thought process. 

This amounts to asking people to adopt a metacognitive 
approach in orderto become more knowledgeable 
about their own thinking and to be better able to 
regulate it. The framework consists of fourquestions. 

1 

What is the goal? 

2 

What is known? 

3 

Which thinking skills will getyou to yourgoal? 

4 

Have you reached yourgoal? 

Recognising that it takes time and conscious effort 
to develop the attitude and skills of a critical thinker 
(to the point where the approach becomes habitual), 
Halpern recommends that teachers provide many 
opportunities to use critical thinking and that 
teachers and learners alike value the development 
of the following sixcritical thinking dispositions: 

■ willingness to plan 

■ flexibility (open-mindedness) 

■ persistence 

■ willingness to self-correct 

■ being mindful (metacognitive monitoring) 

■ consensus seeking. 



Table 3 
An example of 
one of the critical 
thinking skills 
specified byHalpern 



Skill 


Description 


Example of use 


Avoiding the 
entrapment bias 


Entrapment occurs 
when a course of action 
requires additional 
investments beyond 
those already made 


Shana decides to 
stick with her boyfriend 
who treats her badly 
because she has already 
invested several years 
in the relationship 



Evaluation: scope and structure 

As Halpern no longerclaims that her reviews of critical 
thinking skills constitute a taxonomy, we cannot 
evaluate it as such. We have included it here because 
it can certainly be described as a framework and 
because Halpern’s work is wide in scope and rich in 
detail. She also makes use of a superordinate guiding 
principle - metacognition. 

When considered in relation to Marzano’s 
classificatoryframework (2001a, 2001b), Halpern's 
reviews address the cognitive and metacognitive 
systems. In particular there is a close correspondence 
between some of Halpern’s main categories and 
Marzano’s knowledge utilisation categories. When 
compared with Bloom’s taxonomy (1956), Halpern’s 
reviews coverall aspects of the cognitive domain 
with the exception of application. This is not because 
the use of procedures is excluded from the skill areas 
she covers, but because of her emphasis on critical 
thinking which, unlike most routine application, 
is essentially metacognitive in nature. There is one 
sense, however, in which she is extremely interested 
in application, not as a separate category, but for its 
importance in all skill areas. Indeed, her main focus 
is on the conscious application of a 'plan-do-review' 
or'plan-decide-act-monitor-evaluate’ cycle to all thinking 
skills and orchestrated uses of skills. 



Table 4 

Halpern's categorisation 
of critical thinking skills 



Memory skills 

Skills that are needed when learning, during retention and 
at retrieval 

a monitoring your attention 

■ developing an awareness of the influence of stereotypes and other 
beliefs on what we remember 

■ making abstract information meaningful as an aid to 
comprehension and recall 

■ using advance organisers to anticipate new information 

n organising information so that it can be recalled more easily 

■ generating retrieval cues at both acquisition and retrieval 
s monitoring how well you are learning 

■ using external memoryaids 

■ employing keywords and images, rhymes, places, and first letters, 
as internal memoryaids 

■ applying the cognitive interview techniques (Geiselman and 
Fisher 1985) 

■ developing an awareness of biases in memory 
Thought and language skills 

Skills that are needed to comprehend and defend against the 
persuasive techniques that are embedded in everyday language 

■ recognising and defending against the use of emotional and 
misleading language 

■ detecting misuse of definitions and reification 

■ understanding the use of framing with leading questions and 
negation to bias the reader 

■ using analogies appropriately 

■ employing questioning and paraphrase as a skill forthe 
comprehension of text and oral language 

■ producing and using a graphic representation of information 
provided in prose form 

Deductive reasoning skills 

Skills used to determine if a conclusion is valid - ie it must be true 
if the premises are true 

■ discriminating between inductive and deductive reasoning 

■ identifying premises and conclusions 

■ reasoning with 'if, then' statements 

■ using linearordering principles 

■ avoiding the fallacies of denying the antecedent and confirming 
the consequent 

■ using tree diagrams with branches and nodes to represent 
information 

Argument analysis skills 

Skills that are needed to judge how well reasons and evidence 
support a conclusion, including considering counter-evidence, 
stated and unstated assumptions, and the overall strength 
of the argument 

■ identifying premises (reasons), counterarguments and conclusions 
s making strong arguments that show good thinking and 

communication skills 

■ judging the credibility of an information source and judging the 
difference between expertise in factual matters and in value 
matters 

■ understanding the difference between opinion, reasoned judgement 
and fact 

■ recognising and avoiding common fallacies, such as straw person, 
appeals to ignorance, slipperyslope, false dichotomy, guilt by 
association, and arguments against the person 

■ identifying psychological effects on reasoning 

■ remembering to consider what could be missing from an argument 



Skills in thinking as hypothesis testing 

The skills used in scientific reasoning - the accumulation 

of observations, formulation of beliefs or hypotheses, and then 

using the information collected to decide if it confirms or disproves 

the hypotheses 

■ recognising the need for, and using, operational definitions 

■ understanding the need to isolate and control variables in order 
to make strong causal claims 

■ checking for adequate sample size and possible bias in sampling 
when a generalisation is made 

■ being able to describe the relationship between anytwo variables 
as positive, negative, or unrelated 

■ understanding the limits of correlational reasoning 

Likelihood and uncertainty critical thinking skills 

The correct use of objective and subjective estimates of probability 

■ recognising regression to the mean 

■ understanding and avoiding conjunction errors 

■ using base rates to make predictions 

■ understanding the limits of extrapolation 

■ adjusting risk assessments to account forthe cumulative nature 
of probabilistic events 

■ thinking intelligently about unknown risks 

Decision-making skills 

The skills involved in the generation and selection of alternatives 
and in judging among them 

■ framing a decision in several ways to consider different sorts 
of alternative 

■ generating alternatives 

■ evaluating the consequences of various alternatives 

■ recognising the bias in hindsight analysis 

■ using a decision-making worksheet 

■ avoiding the entrapment bias 

■ seeking disproving evidence 

■ awareness of the effects of memoryon decisions 

Problem-solving skills 

Skills needed to identifyand define a problem, state the goal 
and generate and evaluate solution paths 

■ restating the problem and the goal to consider different sorts 
of solution 

■ recognising the critical role of persistence 

■ using a quality representation of a problem (eg graphs, trees, 
matrices, and models) 

■ understanding world-view constraints 

■ selecting the best strategy for the type of problem 

■ activelyseeking analogies 

Skills for creative thinking 

■ redefine the problem and goal (in several different ways) 

■ find analogies (across different domains of knowledge) 

■ list relevant terms 

■ brainstorm (without censoring orevaluation) 

■ generate and use lists of ways in which a solution can vary 

■ list attributes 

■ listthe positive, negative and interesting attributes of various 
solutions 

■ visualise from other perspectives 
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Halpern deals almost incidentally with the affective 
aspects of thinking, as illustrated by the cognitive 
emphasis in her treatment of creative thinking and from 
her rather limited list of critical thinking dispositions 
(compared with those proposed by Costa, Ennis, Paul 
or Perkins, J ay and Tishman 1993). She takes conative 
aspects more seriously, as can be seen from her use 
of the terms 'willingness' and 'persistence'. 

The inclusion of memory as an area where 
critical thinking, in the form of self-knowledge and 
self-regulation, is important is worthy of comment, 
since it is ignored by many othertheorists, 
despite being a well-established area of research 
in cognitive psychology. 

Halpern does not claim to have provided comprehensive 
lists of critical thinking skills. It is possible to identify 
many gaps in her lists, some in relation to other 
work in the same area (eg Allen, Feezel and Kauffie’s 
(1967) more detailed treatment of argument analysis 
skills), and even in relation to other lists provided 
by Halpern herself in the same chapter [eg 'seeking 
converging validityto increase yourconfidence in 
a decision' and 'considering the relative "badness" 
of different sorts of errors’, (1997, 158)]. There are 
also some examples of skills included in the 1992 
taxonomy, but not appearing explicitly in the 1997 
reviews [eg 'solving problems with proportional and 
combinatorial (systematic combinations) reasoning' 
(1994, 34)]. Moreover, as the 'plan-do-review' feature 
of critical thinking is presented as an overarching 
principle, these types of skill are in many cases 
missing from each of the review sections. This is 
especiallytrue of her treatment of problem solving. 

It is not surprising thatthere is a considerable amount 
of overlap between sections, since complex sets 
of skills are being described. Forexampie, decisions are 
often taken in the course of problem solving orcreative 
thinking, and deductive and inductive reasoning as 
well as the use of analogy can be valuable in many 
types of thinking. 

One point of difference between Halpern’s 1992 
taxonomy and her 1997 reviews is that in the earlier 
version, deductive reasoning was not separated from 
argument analysis skills. At first sight, the separate 
treatment given to deduction seems inconsistent with 
Halpern's general exclusion of mechanical procedural 
applications (such as the rules of formal logic and the 
analysis of syllogisms). To some extent it is, but in 
her book (1997), Halpern considers the psychological 
factors which militate against human beings being 
as logical as computers, and the need to guard critically 
against psychologically-induced distortions. 



Overall, Halpern provides a detailed, but not 
comprehensive account of thinking skills within 
the cognitive domain. She asks the readerto apply 
a superordinate organising principle - metacognition - 
in orderto develop an effective critical thinking 
approach. This is virtually equivalent to defining critical 
thinking as 'mindful thinking’. 

Evaluation: theory and values 

Halpern is a strong believer in the application 
of rational methods in problem solving, including the 
use of controlled experiments. She points to the need 
for people to learn how to learn and to be critically 
selective in responding to the barrage of information 
(including advertisements and political rhetoric) around 
them (1997). She argues thatteaching and assessing 
critical thinking will improve the quality of teaching 
and learning at college level and will increase social 
capital and economic competitiveness (1994). These 
are pragmatic arguments, in support of which she cites 
a number of studies to illustrate the transferability 
of critical thinking skills. 

More than any author whose work we have reviewed, 
Halpern has succeeded in her intention to translate 
theory and research from cognitive psychology into 
a form where it can be useful in everyday life and, more 
importantly, can help people to cope responsibly with 
the important decisions on which the future of the world 
depends. In doing so, she has also drawn on relevant 
sources outside psychology, citing, forexampie, 
the work of Polya (1945) on problem solving, Norris 
and Ennis (1989) on the assessment of arguments 
and de Bono (1976) on creative thinking. 

Evaluation: communicability and practical 
relevance 

The communicability of Halpern’s writing about critical 
thinking is demonstrated by the fact that there have 
been fourfurther versions of her book Thought and 
knowledge since 1984. She writes in an accessible style 
for a student audience as well as for a more general 
adult readership, and makes use of many engaging 
real-life illustrations to explain concepts and develop 
arguments. She has also produced up-to-date teaching 
material to accompanythe main text (2002). 

Halpern explicitly designed a wide range of problems 
and exercises to maximise transferacross subject 
boundaries and from academic to other settings. 
Materials of this kind can be included in free-standing 
modules orcan be built into subject teaching forpost-16 
learners. Forexampie, Halpern's 'skills in thinking as 
hypothesis testing’ category includes content 
traditionallytaught in research methods courses 
in a range of disciplines. This could be taught within 
a particulardiscipline, on an interdisciplinary basis, 
oras part of a free-standing module on critical thinking. 



3.4.3 

Paul's model of critical thinking 

Description and intended use 

Richard Paul is a passionate reformer who sees 
educational transformation as the keyto solving social 
and economic problems. He is part of a philosophical 
movement that calls for the engagement of each 
individual in thinking abouttheirbeliefs and a change 
in teaching methods to make this possible. Paul (1989, 
15) views lecture-orientated, didactic and drill-based 
models of instruction as instrumental in producing 
'students who do not learn howto work by orthink 
forthemselves’. Apart from his work with educators, 
he has provided critical-thinking seminars forbusiness 
executives, theologians, doctors, nurses and 
otherprofessionals. 

Paul’s (1993, 33) definition of critical thinking gives 
an insight into his philosophy of education. 

Critical thinking is disciplined self-directed thinking 
which exemplifies the perfections of thinking 
appropriate to a particularmode ordomain of thinking. 

It comes in two forms. If the thinking is disciplined to 
serve the interests of a particularindividual orgroup, 
to the exclusion of other relevant persons and groups, 

I call it sophistic or weak sense critical thinking. If the 
thinking is disciplined to take into account the interests 
of diverse people or groups, I call it fair-minded orstrong 
sense critical thinking. In thinking critically, we use 
ourcommand of the elements of thinking to adjustour 
thinking successfully to the logical demands of a type 
or mode of thinking. As we come to habitually think 
critically in the strong sense we develop special traits 
of mind: intellectual humility, intellectual courage, 
intellectual perseverance, intellectual integrity, and 
confidence in reason. Asophistic orweaksense critical 
thinker develops these traits only in a restricted way, 
consistent with egocentric and sociocentric 
commitments. 

Paul’s model of critical thinking has evolved over 
a numberof years and remains a work in progress. 

It is illustrated in Figure 4, which includes Paul’s 
original 35 elements of critical thought, which have 
subsequently been renamed 'a strategy list for 
redesigning lessons'. The original 35 elements 
contained nine affective strategies which Paul called 
'the traits of a disciplined mind’. These traits have 
been retained in Paul’s most recent model of critical 
thinking (1993) underthe heading of 'intellectual traits’. 
The micro skills and macro abilities no longerfeature 
in the latest critical thinking model. 



The current model has four parts: elements of reasoning 
(sometimes referred to as elements of thought), 
standards of critical thinking, intellectual abilities 
and intellectual traits. The first three categories focus 
on what is essential to critical thinking, while the last 
dimension focuses on what it is to be a critical thinker. 

Elements of reasoning 

This is what Paul refers to as the 'parts' of thinking 
orthe fundamental structures of human thought. 

He maintains that these eight elements are always 
present in human thinking and thatthe ability to 
recognise these elements of reasoning is essential 
to critical thinking. Paul and Elder (2001, 53) 
attempt to condense this interrelated set of eight 
elements in the following statement: 

Whenever you are reasoning, you are trying to 
accomplish some purpose, within a point of view, 
using concepts or ideas. You are focused on some 
question, issue or problem, using information to come 
to conclusions, based on assumptions, all of which 
has implications. 

Standards of critical thinking 

The standards in Paul’s model are an attempt to identify 
what constitutes the quality component of critical 
thinking. Unlike the elements of reasoning which Paul 
claims to be universal, the list of standards seeks 
to encompass those that are the mostfundamental. 

In orderto learn to reason well, it is necessary to gain 
mastery of both the elements of reasoning and the 
standards of critical thinking described in Figure 4. 

Intellectual abilities 

According to Paul, an ability is composed ofa process, 
plus an object, plus a standard. Someone can have 
the ability to drive (process) a truck (object) safely 
(standard). Nosich (2000) proposes that an intellectual 
ability would be the ability, for instance, to identify 
(process) a conclusion (object) accurately (standard). 

In Paul's model, abilities (higher-orderthinking skills) 
rest on a prior understanding of the elements and 
standards of critical thinking. 
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Intellectual traits 

The final dimension of Paul's model focuses on what 
it is to be a critical thinker. He has identified a number 
of affective traits that he considers to be essential 
to 'strong sense’ critical thinking. These are not things 
a person does, but describe how a person is orcan 
be. These 'traits of a disciplined mind' (see below) are 
what Paul calls the affective and moral dimensions of 
critical thinking. Paul (1991) claims thatthere are many 
ways in which teachers can foster these traits of mind. 
To do this successfully, he advocates a complete 
re-conceptualisation of the nature of teaching and 
learning in every context of school life, involving a move 
from a didactic to a critical theory of education. 

Intellectual humility 

Havinga consciousness of the limits of one’s 
knowledge, including a sensitivityto circumstances 
in which one’s native egocentricity is likely to 
function self-deceptively; sensitivityto bias, prejudice 
and limitations of one’s viewpoint. Intellectual 
humility depends on recognising that one should not 
claim more than one actually knows. It does not imply 
spinelessness orsubmissiveness. It implies the lack of 
intellectual pretentiousness, boastfulness, orconceit, 
combined with insight into the logical foundations, 
or lack of such foundations, of one's beliefs. 

Intellectual courage 

Having a consciousness of the need to face and fairly 
address ideas, beliefs or viewpoints towards which 
we have strong negative emotions and to which we have 
not given a serious hearing. This courage is connected 
with the recognition that ideas considered dangerous 
orabsurd are sometimes rationallyjustified (in whole 
or in part) and that conclusions and beliefs inculcated 
in us are sometimes false ormisleading. To determine 
for ourselves which is which, we must not passively and 
uncritically 'accept' what we have 'learned'. Intellectual 
courage comes into play here, because inevitably we 
will come to see some truth in some ideas considered 
dangerous and absurd, and distortion orfalsityin some 
ideas strongly held in oursocial group. We need courage 
to be true to ourown thinking in such circumstances. 
The penalties fornon-conformitycan be severe. 

Intellectual empathy 

Having a consciousness of the need to imaginatively 
put oneself in the place of others in order to genuinely 
understand them, which requires the consciousness 
of ouregocentric tendencyto identifytruth with our 
immediate perceptions of long-standing thought or 
belief. This trait correlates with the abilityto reconstruct 
accurately the viewpoints and reasoning of others 
and to reason from premises, assumptions, and ideas 
other than ourown. This trait also correlates with the 
willingness to rememberoccasions when we were wrong 
in the past despite an intense conviction that we were 
right, and with the abilityto imagine ourbeing similarly 
deceived in a case at hand. 



Intellectual integrity 

Recognition of the need to be true to one's own thinking; 
to be consistent in the intellectual standards one 
applies; to hold one's self to the same rigorous 
standards of evidence and proof to which one holds 
one’s antagonists; to practise what one advocates 
forothers; and to honestlyadmitdiscrepancies and 
inconsistencies in one’s own thought and action. 

Intellectual perseverance 

Having a consciousness of the need to use intellectual 
insights and truths in spite of difficulties, obstacles, 
and frustrations; firm adherence to rational principles 
despite the irrational opposition of others; a sense 
of the need to struggle with confusion and unsettled 
questions overan extended period of time to achieve 
deeperunderstanding orinsight. 

Faith in reason 

Confidence that, in the long run, one’s own higher 
interests and those of humankind at large will 
be best served by giving the freest play to reason, 
by encouraging people to come to theirown conclusions 
bydevelopingtheirown rational faculties; faith that, 
with properencouragement and cultivation, people 
can learn to think forthemselves, to form rational 
viewpoints, draw reasonable conclusions, think 
coherently and logically, persuade each other by 
reason and become reasonable persons, despite 
the deep-seated obstacles in the native character 
of the human mind and in societyas we know it. 

Fairmindedness 

Having a consciousness of the need to treat all 
viewpoints alike, without reference to one's own feelings 
orvested interests, orthe feelings or vested interests of 
one's friends, community or nation; implies adherence 
to intellectual standards without reference to one’s own 
advantage orthe advantage of one's group. 

Elderand Paul (1998, 34) 

Paul presents the traits of the disciplined mind 
as ideals to strive towards. He outlines six stages 
of critical thinking, moving from the unreflective thinker 
to the masterthinker. Master thinkers are: conscious 
ofthe working of theirmind, highly integrated, mentally 
powerful, logical, far-sighted, deep, self-correcting and 
mentally free. According to Paul, masterthinkers will 
only emerge when society begins to value and reward 
these qualities of thinking. Given the extent of deep 
social conditioning, he believes it unlikely that anyone 
currently meets his definition of masterthinker. 



Figure 4 
Paul's model 
of critical thinking 



Cognitive strategies - macro abilities 

■ Refining generalisations and avoiding 
oversimplifications 

■ Comparing analogous situations: transferring 
insights to new contexts 

■ Developing one's perspective: creating or exploring 
beliefs, arguments, ortheories 

■ Clarifying issues, conclusions, or beliefs 

■ Clarifying and analysing the meanings of words 
or phrases 

■ Developing criteria forevaluation: clarifying values 
and standards 

■ Evaluating the credibility of sources of information 

■ Questioning deeply: raising and pursuing root 
or significant questions 

■ Analysing orevaluating arguments, interpretations, 
beliefs, ortheories 

■ Generating or assessing solutions 

■ Analysing orevaluating actions or policies 

■ Reading critically: clarifying or critiquing texts 

■ Listening critically: the art of silent dialogue 

■ Making interdisciplinary connections 

■ Practising Socratic discussion: clarifying and 
questioning beliefs, theories, or perspectives 

■ Reasoning dialogically: comparing perspectives, 
interpretations, 

ortheories 

■ Reasoning dialectically: evaluating perspectives, 
interpretations, 

ortheories 



Cognitive strategies - micro skills 

■ Comparing and contrasting ideals with 
actual practice 

■ Thinking preciselyabout thinking: 
using critical vocabulary 

■ Noting significant similarities and differences 

■ Examining orevaluating assumptions 

■ Distinguishing relevant from irrelevant facts 

■ Making plausible inferences, predictions 
or interpretations 

■ Giving reasons and evaluating evidence and 
alleged facts 

■ Recognising contradictions 

■ Exploring implications and consequences 



Strategy lists for redesigning lessons 
(formerly 35 elements of critical thinking) 



Taxonomyof Socratic questions 

■ Questions of clarification 

■ Questions that probe assumptions 

■ Questions that probe reasons and evidence 

■ Questions about viewpoints or perspectives 

■ Questions that probe implications and consequences 

■ Questions about the question 



Intellectual integrity 
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Affective strategies 

■ Thinking independently 

a Developing insight into egocentricityor 
sociocentricity 

a Exercising fair-mindedness 

■ Exploring thoughts underlying feelings and feelings 
underlying thoughts 

■ Developing intellectual humilityand suspending 
judgement 

■ Developing intellectual courage 

a Developing intellectual good faith or integrity 

■ Developing intellectual perseverance 

a Developing confidence in reason 





1 

Elements of reasoning 

■ To be clearaboutthe problem orthe question at hand 

■ To be clearaboutthe goal or purpose 

■ To be clearaboutthe point of view or fra me 
of reference 

■ To be clear about assumptions 

■ To be clearaboutthe claims we are making 

■ To be clearaboutthe reasons or evidence upon which 
we base ourclaims 

■ To be clear about our inferences and line of reasoning 

■ To be clear about the implications and consequences 
that follow from our reasoning 

2 

Standards of critical thinking 

■ Clarity 

■ Precision 

■ Specificity 

■ Accuracy 

■ Relevance 

■ Consistency 

■ Logical 

■ Depth 

■ Completeness 

■ Significance 

■ Adequacy (for purpose) 

■ Fairness 
3 

Intellectual abilities 

Composed of a process, object and a standard, 
eg drive a truck safely = drive (process) truck (object) 
safely (standard) 



Essential to critical thinking 

4 

Intellectual traits 

Affective traits as described in 'Traits of the 
disciplined mind’ 



Essential to what it is to be 
a critical thinker 




Evaluation: scope and structure 

Paul’s model of critical thinking takes account 
of cognitive, affective and conative components. 

He is aware of the importance of being sensitive to 
the circumstances in which thinking occurs. His lists 
of abilities and traits do not have any significant 
omissions when compared with those of Ennis and 
Perkins, JayandTishman (1993). Although Paul does 
not use the term 'metacognition’, his account of 
intellectual integrity does recognise it, as it stresses 
applying the same standards to one's own thinking 
as one does to other people’s. 

Nosich (2000) believes that it is because Paul's 
model of critical thinking is concept-based (as opposed 
to having rules, procedures or steps to follow), that 
it is effective in curriculum development. The model 
is extremely flexible, applicable to any subject matter 
and to any level of thinking. Nosich uses Paul’s concept 
of evidence to illustrate his point. Asking a student 
to 'identify the evidence a conclusion is based upon’ 
is a critical thinking step. The abilityto do it well is 
a higher-order thinking skill, but is limited in that there 
are many other things students should be able to do 
with evidence; forexample: 

■ evaluate the evidence 

■ clarifythe evidence 

■ realise the need for more evidence 

■ contrast one account of the evidence with another 
account 

■ see how the evidence arises out of certain background 
assumptions 

■ understand the nuances of the evidence 

■ ask howthis evidence in this case fits in with 
the actual observations. 

This list, which is by no means exhaustive, gives 
an insight into what Paul means by his concept 
of evidence. The goal is to make the concept of evidence 
an essential part of all thinking processes. Nosich 
(2000) believes thatthe abilityto think in terms of the 
concept of evidence allows one to think about evidence 
in a range of settings, at any level of expertise and 
to gain further insight into the concept of evidence. 

Nosich maintains that Paul’s standards of critical 
thinking also function as concepts. We are once 
again asked to consider the skill 'identify the evidence 
a conclusion is based upon'. Nosich claims that implicit 
in the skill is using the standards of accuracy, clarity, 
adequacyand relevance. When one adopts these 
concepts in relation to the concept of evidence, there 
is an increased abilityto think in terms of the standards 
across the other elements of reasoning. 



Evaluation: theory and values 

Thayer-Bacon (1998) notes that until recently, critical 
thinking, with its valuing of reason overother qualities, 
has had few detractors in the literature. She claims 
(1998, 125): 

This valuing of reason over other tools is not new 
with current critical thinking philosophers, it reflects 
a Euro-western cultural bias that can be traced 
all the way back to ancient Greece. The valuing 
of reasoning overall otherabilities is also a gender 
bias, as reasoning (the abilityto think logically) 
has been considered predominantly a male ability 
in the Euro-western world, while intuition, imagination, 
and emotional feelings have been associated with 
women’s abilities. 

Paul is one of a number of philosophers whom 
Thayer-Bacon criticises fortheirbelief in rational 
thought as the dominant mode of thinking within 
a critical thinking framework. It is evident from Paul’s 
definition of critical thinking and his other work, 
including the development of a taxonomy of Socratic 
questioning, that he sits firmly in the rationalist camp. 
For Paul, the spirit of critical thinking is to have the 
confidence in one's abilityto figure out the logic 
of anything. He acknowledges that there have been 
criticisms of critical thinking for being too Western 
in its orientation, not dealing with creativity, ignoring 
the role of emotion in thought and failing to address 
feministorsociological insights. However he claims 
that previous attempts to widen the scope of critical 
thinking to accommodate these concerns meant 
sacrificing some of the rigour and exactitude found 
in formal and informal logic courses. Paul proposes that 
his philosophy is a response to the problems inherent 
in the critical thinking courses initiated in the 1970s 
with theiremphasis on pure logic, and to subsequent 
criticisms of these courses as outlined above. 

Paul's majorcontribution to the area of critical thinking 
would appearto be his ideas of 'weak' versus 'strong 
sense’ thinking. The latter is what Paul refers to as 
the abilityto discover and contest one's own egocentric 
and socio-centric habits of thought. Paul (1991, 77) 
claims that these nine traits of thought, which are moral 
commitments and intellectual virtues, transfer thinking 
from 'a selfish, narrow-minded foundation to a broad 
open-minded foundation'. 



Evaluation: communicability and practical 
relevance 

The clearlanguage and writing style are commendable 
in Paul's work. It is unlikelythat his writing would prove 
a barrier to anyone with an interest in critical thinking. 
Practitioners teaching courses in critical thinking 
mayfind Paul’s work inspirational, given its emphasis 
on 'strong sense' thinking. 

Paul will find less resonance with those teachers 
not involved with critical thinking courses. While 
Paul offers some teaching strategies to support the 
development of pupils' thinking, his aim is not to tinker 
with classroom practice byproposing thinking skills 
programmes. His goal is a reworking of education, 
a move from a didactic to a critical theoryof education, 
one where students construct knowledge through 
the application of theirown logic ratherthan through 
teachertransmission. He calls fora re-evaluation of 
what is judged important in both education and society. 

3.5 

Evaluation of cognitive structure and/or 
development frameworks 

King and Kitchener (1994) make epistemological 
(theoryof knowledge) claims about how the nature 
of knowledge changes as people progress through their 
stages, while Pintrich provides a theoretical framework 
for understanding self-regulation. In Appendix 2, we 
present brief evaluative summaries of three models 
of intelligence - those of Guilford (1958), Gardner 
(1983) and Carroll (1993); together with three 
developmental frameworks - those of Perry (1970), 
Belenky et al. (1986) and Koplowitz (1987). 

3.5.1 

King and Kitchener's model of reflective judgement 

Description and intended use 

King and Kitchener propose a seven-stage model of 
reflective judgement in their book Reflective judgment: 
understanding and promoting intellectual growth 
and critical thinking in adolescents and adults (1994). 
The model is aimed at those who work in the area 
of critical thinking at college level, particularly in regard 
to its development and assessment, though the authors 
also indicate that it should be of value for use in schools 
and in otheradult learning contexts. The model is 
based on Dewey’s (1933, 1938) conception of reflective 
thinking and the epistemological issues resulting from 
attempts to resolve 'ill-structured problems’. It draws 
on otherwork, such as Fischer's (1980) skill theoryand 
is related to the work of Perry (1970) and Baron (1985). 

It is summarised in Table 5 opposite. 



Table 5 

King and Kitchener's 
seven-stage model 



Stage 1 


Knowing is limited to single concrete 
observations: whata person observes 
is true. Discrepancies are notnoticed 


P re-reflective 
thought 


Stage 2 


Two categories for knowing: right answers 
andwronganswers. Good authorities 
have knowledge: bad authorities lack 
knowledge. Differences can be resolved 
by more complete information 




Stage 3 


In some areas, knowledge is certain 
and authorities have knowledge. 
Inotherareas, knowledge is temporarily 
uncertain: onlypersonal beliefs can 
be known 




Stage 4 


The conceptthat knowledge is unknown 
in several specific cases can lead to the 
abstract generalisation that knowledge is 
uncertain. Knowledge and justification 
are poorly differentiated 


Quasi-reflective 

thought 


Stage 5 


Knowledge is uncertain and must 
be understood within a context: 
thus justification is context-s pecif ic. 
Knowledge is limited bythe perspective 
of the person who knows 




Stage 6 


Knowledge is uncertain, but constructed 
by comparing evidence and opinion 
on different sides of an issue or across 
contexts 


Reflective 

thought 


Stage 7 


Knowledge is the outcome of a process 
of reasonable enquiry. This principle 
is equivalentto a general principle across 
domains. Knowledge is provisional 





Evaluation: scope and structure 

King and Kitchener(1994) distinguish reflective 
judgement from logical, verbal and moral reasoning. 
Theirmodel is based on 15 years of theory building 
and empirical research into the development 
of reflective judgement in late adolescence and middle 
adulthood. Itshows further development from their 
original study of reflective judgement (Kitchenerand 
King 1981). On the basis of more than 30 studies, they 
claim - we believe fairly- thatthe model is complex, 
inclusive and integrated, with qualitative differences 
that are stable across domains observable in reasoning 
about knowledge. 

Hoferand Pintrich (1997) have pointed to structural 
similarities between King and Kitchener’s model, 

Perry’s account (1968, 1970) of intellectual and ethical 
development and the work of Belenky et al. (1986) 
on 'women’s ways of knowing'. King and Kitchener’s 
concept of stages is heavily influenced by Piaget’s 
genetic epistemology and is also not unlike the model 
on which Biggs and Collis (1982) base their 
SOLO taxonomy. 

There are two related issues which King and Kitchener 
do notfullyaddress. The first is the extentto which 
reflective judgement, as assessed by being asked 
to solve a set of ili-structured problems, relates 
to thinking and performance in otherfields - personal 
and professional. The second issue is a concern 
about whether the 'stage theory’ is genuinely about 
broad epistemological assumptions and beliefs; 
or whether, in a series of reflective judgement 
interviews, respondents learn to provide more 
sophisticated answers to a specific set of increasingly 
familiarquestions. 

Evaluation: theory and values 

The model identifies a progression of seven distinct 
sets of judgements about knowledge and how 
knowledge is acquired. Each set has its own logical 
coherence and is called a stage, with each successive 
stage 'posited to represent a more complex and 
effective form of justification, providing more inclusive 
and better integrated assumptions for evaluating and 
defending a point of view’ (King and Kitchener 1994, 

13). Individuals pass through these stages in the order 
specified, though they may operate across a range 
of stages at any point in time. This still leaves questions 
about how individuals progress through the stages 
and about the relationship between maturation, 
education and culture. 

King and Kitchenerhave studied the relationship 
between reflective judgement and moral reasoning. 
While they endorse the view thatthe college 
experience should provide an education in character 
development, they see progress through the seven 
stages of development in reflective judgement as 
furnishing necessary, but not sufficient, conditions 
forcorresponding progress in moral reasoning. 



The model of reflective judgement is a coherent, 
well-argued and extensively researched account 
of the development of epistemological reasoning, 
though there are some issues that remain unresolved. 
The authors acknowledge limitations in theirsample 
selection, which may not make it representative of 
a larger population outside US mid-western high-school 
and college students. Also, the epistemological 
assumptions in the final level of 'reflective thought' 
in stages 6 and 7 may be less prevalent in some 
cultures (Bidell and Fischer 1992). 

There is evidence from othersources that assumptions 
about knowledge do alteraccording to the subject 
context (eg Schoenfeld 1992). This suggests that 
the confidence of the authors that students’ scores 
on subject-based problems are almost identical 
to standard reflective judgement interview scores 
may need further investigation across disciplines. 

Evaluation: communicability and practical 
relevance 

Chapter9 of King and Kitchener’s book contains 
explicit recommendations forteaching, using the 
reflective judgement model as a 'heuristic tool' 
to help teachers and othereducators to develop 
courses oractivities to help learners to think more 
reflectively and make more reasoned judgements. 

The basis for using the model is set out in a series 
of assumptions, supporting activities to develop 
personal relevance and a detailed breakdown of each 
of the stages 2-7 with characteristics, instructional 
goals, difficult tasks, sample activities orassignments 
and developmental support. These are sufficiently 
clearand detailed to be applicable to educational 
practice in a range of settings. The main challenge 
in using the model is how to develop a clear 
understanding of each of the seven stages and how 
to recognise learners’ behaviours at each stage. 

3.5.2 

P intrich's general framework for self-regulated 
learning 

Description and intended use 

Pintrich (2000) defines self-regulated learning 
(SRL) as '... an active, constructive process whereby 
learners set goals fortheir learning and then attempt 
to monitor regulate and control theircognition, 
motivation and behaviour, guided and constrained 
bytheirgoals and the contextual features in the 
environment.’ Table 6 opposite, reproduced from 
Pintrich’s chapter displays a framework forclassifying 
the different phases of, and areas for regulation. It can 
be seen that he differentiates between fourdomains, 
cognition, motivation, behaviourand context. 
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Table 6 

Phases and areas for 
self-regulated learning 



Areas for regulation 



Phases 


Cognition 


Motivation/affect 


Behaviour 


Context 


1 

Forethought, planning 
and activation 


Target goal setting 
Priorcontent knowledge 
activation 


Goal orientation adoption 
Efficacy judgements 
Ease of learning 


Time and effort planning 
Planning for 
self-observations of 


Perceptions of task 
Perceptions of context 




Metacognitive knowledge 
activation 


judgements; perceptions 
of task difficulty 


behaviour 








Task value activation 










Interest activation 






2 

Monitoring 


Metacognitive 
awareness and 
monitoring of cognition 


Awareness and 
monitoring of motivation 
and affect 


Awareness and 
monitoring of effort, time 
use, need for help 
Self-observation 
of behaviour 


Monitoring and changing 
task and context 
conditions 


3 

Control 


Selection and adaptation 
of cognitive strategies for 
learning, thinking 


Selection and adaptation 
of strategies for 
managing motivation and 
affect 


Increase/ decrease effort 
Persist, give up 
Help-seeking behaviour 


Change or renegotiate 
task 

Change or leave context 


4 

Reaction and reflection 


Cognitive judgements 
Attributions 


Affective reactions 
Attributions 


Choice behaviour 


Evaluation of task 
Evaluation of context 



Regulation of cognition 

Although cognitive skills are clearly central to 
thinking skills, they also playa part in the regulation 
of motivation, affect, behaviourand context. 

Cognitive planning and activation 

The framework proposes three general types of planning 
oractivation. 

■ Target goal setting: once task-specific goals have been 
identified, they can then be used to guide cognition 
and monitoring processes. These goals may need to be 
adjusted orchanged duringtask performance as part 
of the monitoring, control and reflection processes. 

■ Prior content knowledge activation: refers to when 
learners actively search theirmemoryforrelevant prior 
knowledge (both content and metacognitive) before 
performing the task. 

■ Metacognitive knowledge activation: metacognitive 
task knowledge concerns understandings about 
the influence of different types and forms of task 
upon cognitive demands (eg the more information 
that is provided, the easier the task becomes). 
Knowledge of strategy variables concerns those 
procedures that might help with cognitive processes 
such as memorising and reasoning. As with prior 
content knowledge, this activation can be automatic, 
can be prompted by particularfeatures of a given task 
orcontext, orcan be employed in a more controlled 
and conscious fashion. 



Cognitive monitoring 

This involves both being aware of and monitoring one’s 
cognition, so it closely resembles what has traditionally 
been understood by the term 'metacognition'. Pintrich 
contrasts metacognitive knowledge, a relatively static 
elementthatone can claim eitherto have orto lack, 
with metacognitive judgements and monitoring, which 
tend to be more dynamic and relate to processes that 
occuras one undertakes a given task. 

Pintrich highlights two important types of monitoring 
activity: judgements of learning which referto gauging 
personal success at learning something and feeling 
of knowing (eg when one feels one knows something 
but cannot quite recall it - the 'tip of the tongue’ 
phenomenon). 

Cognitive control and regulation 

This refers to the cognitive and metacognitive activities 
that individuals engage in to adapt and change their 
cognition. These are closelytied to monitoring and 
involve the selection and use of various cognitive 
strategies for memory, learning, reasoning, problem 
solving and thinking. Specific techniques include 
the use of visual imagery, mnemonics, advanced 
organisers, and specialised methods of note taking. 
Located within this cell are the strategies that learners 
employto help them with their learning, though, 
as Pintrich indicates, these can be both cognitive 
and metacognitive. 



Cognitive reaction and reflection 

These processes are concerned with personal reflection 
on performance and involve both evaluation and 
attribution. According to Zimmerman (1998), evaluating 
one’s performance is a characteristic of superior 
self-regulation. Similarly, 'good' self-regulators are more 
likely to make attributions for performance outcomes 
that emphasise the influence of the learner's efforts 
and strategies (internal and controllable), rather than 
features beyond the learner’s control, such as a lack 
of ability. 

Regulation of motivation and affect 

While there has been much research examining 
awareness and control of cognition (metacognition), 
there has been much less work concerning similar 
processes with respect to motivation. 

Motivational planning and activation 

Bandura’s work on self-efficacy (1997) has highlighted 
the way an individual’s beliefs about likely success 
in undertaking a particulartask will influence the effort 
subsequently employed. Otherfactors highlighted in 
the motivation literature, such as the value of the task 
to the learner personal interest in the task or content 
domain, and fear of failure, can all be made susceptible 
to student regulation and control in ways that can 
improve the quality of the learning. 

Motivational monitoring 

While the literature in this domain is more sparse 
than that for metacognitive awareness and monitoring, 
it is reasonable to assume that to engage effectively 
in the control and regulation of efficacy, value, interest 
and anxiety, students need first to be consciously 
aware of their beliefs and feelings and to monitor 
them. Approaches that have been employed in the 
scientific literature include attempts to make explicit, 
and subsequently change, students’ maladaptive 
self-efficacy and attributional beliefs. Otherstudies 
have sought to reduce student anxiety by increasing 
coping skills or by showing how one may change 
aversive environmental conditions. 



Motivational control and regulation 

Pintrich lists several methods that students can 
employ to heighten their motivation. These include 
increasing your sense of self-efficacy (eg telling 
yourself that you can succeed in the task), promising 
yourself extrinsic reinforcers (eg going to the pub once 
the assignment has been completed) orattempting 
to heighten intrinsic motivation by restructuring 
the task to make it more interesting. Other strategies 
involve overcoming the tendency to avoid working 
hard because of a concern that poor performance 
may suggest a lack of natural ability, a phenomenon 
known as self-worth protection (Covington 1992). 

Motivational reaction and reflection 

Drawing on attribution theory (Weiner 1986), 

Pintrich suggests that individuals will tryto understand 
the reasons forsuccess orfailure by attributing the 
outcome to such factors as skill, luck and effort. 

A belief that failure occurred through lack of natural 
ability is likely to undermine a student's motivation. 
Attribution retraining, therefore, generally tries to help 
the student to see learning as something that he/ she 
can achieve and control by working hard and using 
effective strategies. Pintrich argues that changing 
attributions for life events will lead to new beliefs that 
will have a bearing on new tasks atthe planning phase. 

Regulation of behaviour 

Behavioural forethought, planning and action 

Pintrich recognises that planning one’s behaviour in 
a purposive manneris ess entia lly a cognitive function. 
However he considers it reasonable to locate student 
attempts to plan theirbehaviourin an intentional 
fashion within the column dealing with behavioural 
regulation. Strategies for learners may include 
various time-management activities (eg planning 
an examination revision schedule or deciding when to 
tackle homework) and self-observation and monitoring 
(eg recording how many new French vocabulary 
words are learned each week, or how many pages 
of a new novel are read). Such information may result 
in furtherplanning and action. 

Behavioural monitoring and awareness 

This involves relating the monitoring of behaviour and 
effort levels in the light of progress made. A student 
may, forexample, plan to work at French course 
assignments on two evenings each week, but may 
find thatthis is insufficient and that additional time 
or greater effort is required. 



LSRC reference 



Section 3 



page 40/ 41 



Behavioural control and regulation 

Here Pintrich refers to the learner’s actual control 
and regulation of behaviour; for example, applying 
persistence and effort. It is important to know when, 
and from whom, to seek help. The skilled learner 
does not wish to become overly dependent on others, 
but does obtain assistance in dealing with particularly 
difficult problems. 

Behavioural reaction and reflection 

This concerns student evaluations of the effectiveness 
oftheircurrentbehaviour(egthatstudying in 4-hour 
blocks is notthe best use of time, orthat putting 
off homework to the last minute often results in poor 
marks). Students mayreact bychanging theirtime 
management, level of effort or, indeed, the course 
they are following. 

Regulation of context 

Contextual forethought, planning and activation 

This concerns the individual’s perception of task and 
context. Students may, for example, have different 
ideas about collaborative learning, the type of answer 
expected, orabout classroom climate. Pintrich points 
out that perceptions may not be highly accurate, yet 
these may still have a majorinfluence. 

Contextual monitoring 

Often students experience difficulty when moving from 
school to college or university because they fail to grasp 
fully the different requirements of adult learning, and 
thus do not adjust their learning strategies or general 
behaviour. Examining and monitoring contextual factors 
that may have bearing upon achievement is therefore 
also important, particularly as such rules, routines and 
criteria are rarely made explicit. 

Contextual control and regulation 

Adult learning provides greateropportunities to control 
and regulate classroom environments, although less 
confident students often prefer to retain a more passive 
role. Outside the lecture hall or workshop, students 
need to take responsibilityforregulating theirstudy 
environment to facilitate their learning (eg removing 
distractions and having an organised study space). 

Contextual reaction and reflection 

This involves the student in evaluating aspects 
of the task orclassroom environment. Evaluations 
mayconcern feelings about engaging in the activities 
concerned, or be more focused upon aspects of the 
student’s learning and achievement. As with cognition 
and motivation, such evaluations can have an important 
influence upon the student’s approach to new tasks 
(at phase 1 -forethought, planning and activation). 



Evaluation: scope and structure 

Pintrich’s synthesis of current SRL theories and 
approaches covers cognition, affect, conation 
and behaviour, as well as contextual factors. He 
successfully synthesises the work of leading theorists, 
notably Boekaerts and Niemivirta (2000), Butlerand 
Winne (1995), Corno (1993), Pintrich and De Groot 
(1990), Pintrich, Wolters and Baxter (2000), Pressley 
(1986), Schunk (1994), Schunk and Zimmerman 
(1994), Winne (1995) and Zimmerman (2000). 

As a synthesis of current theorising, his framework 
differs from other leading theorists such as Boekaerts 
(1997) whose model is divided into cognitive 
and motivational self-regulation; and Zimmerman 
(2000) whose triadic model emphasises personal 
self-regulation (which involves monitoring and regulating 
one’s thoughts and feelings to aid performance), 
behavioural self-regulation (where one observes 
and modifies one's performance), and environmental 
self-regulation (which involves gauging and altering 
one’s current environment). In addressing the 
comprehensiveness of his framework, Pintrich points 
out that not all academic learning falls within the four 
phases outlined, as there are many occasions when 
students learn implicitlyorunintentionally, rather 
than in a focused, self-regulatory fas hion. The phases 
are presented as a heuristic device to organise thinking 
and research on SRL. Theycan also be seen as an 
organising 'plan-do-review' principle forclassifying 
the thinking skills involved in SRL. 

It is also important to recognise that the four 
phases are not necessarily passed through 
in a linearsequence, and often phases mayoperate 
simultaneously. Indeed, Pintrich argues that recent 
research provides little evidence that monitoring 
(Phase 2) and control (Phase 3) are separate in 
people's experiences. The appropriateness of the 
fourth column, context, might seem questionable 
to some, as in many conceptions, self-regulation refers 
onlyto aspects of the self that are being controlled 
orregulated. In line with Zimmerman (2000), however; 
Pintrich's model is based upon a belief that one's 
attempts to monitor and control the environment are 
an important aspect of SRL. Perhaps the most valuable 
part of Pintrich's framework, for those with a good 
knowledge of the field, is his discussion of motivational 
factors - an area where he is a leading theorist. 



Evaluation: theory and values 

Pintrich's framework draws extensively on leading-edge 
psychological research about SRL, a field in which 
he has a substantial reputation. His own contribution 
draws heavily upon goal theory (where he is a major 
contributor). As an empirical researcher, he will 
look for accumulating evidence from well-controlled 
studies and field trials about the benefits for learners 
of pedagogical initiatives designed to encourage 
seif-regulation and independent learning. 

Evaluation: communicability and practical 
relevance 

Pintrich's main focus is essentially academic: theory 
building and empirical research. His classificatory 
framework is a useful introduction to self-regulation 
and is helpful forthose who wish to examine similarities 
and differences between different theoretical models. 
Pintrich also hopes that his formulation will draw 
attention to areas which are currently under-researched 
and may require further investigation. While there 
are likely to be important implications for practitioners, 
he tends to leave the detailed articulation of these 
to others. For practitioners, each of the various cells 
(see Table 6) may need to be fleshed out in greater 
detail through reference to other publications. 
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M aking sense of thinking and learning 



4.1 

Introductory outline 

As we have seen, there have been several attempts 
to produce an integrated framework for understanding 
thinking and learning. Those we called 'all embracing' 
have variouslytaken into account cognitive, affective 
and conative aspects of thinking. Whetherexplicitly 
orimplicitly, theyalso include metacognition as an 
important feature. Here we look at the entire range 
of principles used to establish categories in the 35 
frameworks we have evaluated, and comment on how 
they are used within each of the fourfamily groups. 

We then identify a set of core features and use them 
to develop a new integrated framework suitable for 
post-16 applications, using clearand simple English. 

We also commend three frameworks which best 
represent the field as a whole and which are compatible 
with our integrated framework. (Later; in Section 5.3, 
we make recommendations about using a number 
of otherframeworks forspecific purposes.) 

4.2 

How are thinking skills classified? 

Altogether, we identified a total of 15 different kinds 
of principle that were used in the frameworks we 
evaluated to classifythinking and/orits outcomes. 

As can be seen from the evaluative summaries in 
Appendix2, most frameworks are structured byonly 
two orthree principles and none bya comprehensive 
set. We list the principles used in all 35 frameworks 
under four main headings as follows: 

■ domain 

■ area of experience 

■ subject area 

■ content 

■ types of objective 

■ types of product (including knowledge products) 

■ process 

■ steps/ phases in a sequence orcycle 

■ complexity 

■ level in a hierarchy 

■ type of thinking orlearning 

■ quality of thought/ action 

■ psychological aspects 

■ stage of development 

■ structural features of cognition 

■ strength of dispositions 

■ internalisation of learning 

■ orchestration and control of thinking 

■ level of consciousness. 



It was no surprise to find that the most 
comprehensive frameworks (according to the number 
and range of principles they embody) are members 
of the all-embracing and instructional design families. 
We found eight frameworks which are based on 
a selection of principles from each of the generic 
categories: domain, content, process and psychological 
aspects. Four of these are the all-embracing 
frameworks of Romiszowski (1981), J onassen and 
Tessmer(1996/7), Hauenstein (1998) and Marzano 
(2001a, 2001b). The otherfourare the instructional 
design frameworks of Gagne (1965, 1985), Williams 
(1970), Hannah and Michaelis (1977) and Gouge 
and Yates (2002). It is worth noting that Bloom's 
taxonomic achievements (1956) would have placed 
him in this group, if we had considered his work 
in the affective and psychomotordomains. 

The all-embracing frameworks are more likely than 
others to include some coverage of the affective 
and conative domains as well as cognitive skills. 

The more recent frameworks also include an explicit 
treatment of metacognition. 

Several instructional design frameworks also 
extend beyond the cognitive domain, but members 
of this family are distinctive because their purpose 
is to categorise different kinds of learning objective and 
subject content and/ or how far knowledge and skills 
are internalised. 

Our examination of the classificatory principles 
used in critical/ productive thinking frameworks 
showed thatthe presence orabsence of reflective 
and metacognitive processes and of dispositions with 
conative and affective features are often highlighted 
(as in the frameworks of Ennis, Paul and Lipman; 
see Section 3). However, there are some frameworks 
which are limited to the cognitive/ metacognitive 
domain; forexample, Allen, Feezel and Kauffie (1967) 
and Gubbins (1986). Anotherfeature of critical thinking 
frameworks is that (apart from valuing progress toward 
better critical thinking) theirauthors do not specify 
different kinds of objective (eg global orspecific, 
short-term or long-term). 

Among the frameworks dealing with cognitive 
structure and/ordevelopmentthere are some in 
which classificatory principles from onlyone category 
are used (Belenky et al. 1986, Koplowitz 1987, and 
King and Kitchener 1994). Again, within this type 
of framework, with the exception of Guilford's (1958) 
products dimension, different types of objective 
oroutcome are not identified. 




As at least 15 different kinds of principle have been 
used to classifythinking skills, it is most unlikelythat 
a manageable 'meta -taxonomy’ can be constructed 
which uses all of them. It is perhaps forthis reason 
that Romiszowski (1981) presents three separate 
models: a categorisation of knowledge, a skill cycle 
and a schema of skill categories. Romiszowski, 

Gagne (1965, 1985), Hannah and Michaelis (1977), 
Hauenstein (1998) and M a rza no (2001a, 2001b) 
all come close to providing comprehensive coverage 
of thinking and learning in all areas of experience, 
but all have too many weaknesses to be regarded as 
'off-the-shelf solutions for use in post-16 education 
and training as they stand. 

4.3 

Can several frameworks be fitted into a single 
conceptual system? 

We found that the seven all-embracing frameworks have 
certain structural features in common and share all 
orsome of these with every other framework, especially 
with those directed at instructional design. We saw 
features related to Bloom’s taxonomy of educational 
objectives forthe cognitive domain (1956) in many 
otherframeworks and this gave us a starting point. 

Bloom's taxonomyis basically a three-tier model, which 
we can describe in the following way. Thinking starts 
with and ends with knowledge, whether in the form of 
facts, concepts, rules or skills. Basic thinking consists 
of relatively simple ways of understanding, elaborating 
and using what is known. Higher-orderthinking is 
essentia Ilya learning process which leads to a deeper 
understanding of the nature, justification, implications, 
and value of what is known. 

All the frameworks we have evaluated include 
classifications of higher-order or what (following 
Romiszowski 1981) we prefer to call productive thinking. 
By productive thinking we understand what Bloom refers 
to as analysis, synthesis and evaluation and various 
combinations of these and other processes when they 
lead to a productive outcome. A productive outcome 
maybe a deeper understanding of a topic; a judgement, 
decision orsolution; ora tangible product, such as an 
invention orwork of art. Productive thinking may involve 
planning what to do and say, imagining situations, 
reasoning, solving problems, considering opinions, 
making decisions and judgements, orgenerating new 
perspectives. Critical thinking abilities are subsumed 
bythe more general term productive thinking. 

Productive thinking is supported by critical thinking 
dispositions and related habits of mind. It may become 
so well practised as to be taken forgranted, but 
when energised by feelings and determination, it is, 
in Lipman’s terms (1995), critical, creative and caring. 



Although the three-tier model of knowledge, basic 
thinking skills and productive thinking is useful, 
we found it necessaryto add three more categories. 

In line with several authors, we made a distinction 
between perceiving and recalling knowledge as ways 
of accessing information. In seeking to identify what 
makes forgood thinking and what facilitates meaningful 
learning, many theorists draw attention to reflection 
and motivation as well as to relevant abilities and 
dispositions. We use the terms reflective thinking 
and self-engagement to capture these elements. 

In this way, we arrived at a provisional six-level model 
and found thatthis worked well as a wayof classifying 
the levels of thinking included in each of the frameworks 
described and evaluated in Appendix 2. 

In Table 7 opposite, we show how our six-level model 
corresponds to the National Curriculum categories and 
to the six types of thinking identified by Ashman and 
Conway (1997). It is clear that our model provides more 
detail at the lowerand higher levels, but considerably 
less detail under what we call productive thinking. It is 
also noticeable that the reflective and self-engagement 
aspects of thinking do not feature very strongly in 
the National Curriculum (except in a limited number 
of evaluation objectives). 

Our next step was to try to achieve a rather more 
detailed breakdown, especially of the basic thinking 
skills and productive thinking categories. We decided 
to do this by finding out how practitioners conceptualise 
thinking skills. We carried out a pilot questionnaire 
study with the cooperation of Gateshead College, 
to find out whethercollege teachers tend to share 
a mental model of the thinking required of students; 
and if so, how those thinking skills are grouped. 

This study is fully reported elsewhere 
(Moseley 2003). 

We created a composite 69 -item list of all the thinking 
skills identified in the evaluated frameworks. Each item 
was expressed in two phrases, the second phrase being 
in the simplest possible language, as illustrated in the 
following two items: 

■ Paraphrasing: saying or writing information in your 
own words 

■ Monitoring thinking: checking to see if you are 
'getting somewhere'. 
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Table 7 

Comparison of three 
sets of categories 



National curriculum LS DA project team Ashman and Conway (199 7) 

Information processing Perception 



Knowledge recall Content knowledge 



Basic thinking skills Core thinking skills 



Reasoning 

Enquiry 

Creativity 

Evaluation 



Productive thinking Cognitive processes 

Critical thinking 
Creative thinking 



Reflective thinking Metacognition 



Self-engagement 



Figure 5 

An integrated framework 
for understanding 
thinking and learning 



Strategic and reflective thinking 



Engagement with and management of thinking/ learning, supported by 
value-grounded 5 thinking (including critically reflective thinking) 



▲ 






Cognitive skills 


Information gathering 
Experiencing, recognising 
and recalling 

Comprehending messages 
and recorded information 


Building understanding 
Development of meaning 
(eg byelaborating, 
representing orsharing 
ideas) 

Working with patterns 
and rules 

Concept formation 
Organising ideas 


Productive thinking 
Reasoning 

Understanding causal 
relationships 
Systematic enquiry 
Problem solving 
Creative thinking 



We then asked 37 college teachers to considerthese 
skills in relation to a particularcourse for which they 
are responsible. Using a four-point scale, they indicated 
how strongly each thinking skill features in the course 
as a whole (objectives, teaching and learning, and 
assessment). Ten groups of thinking skills were 
identified through exploratory factoranalysis, which 
we labelled in the following way: 

■ internalising knowledge and skill 

■ working with patterns and rules 

■ concrete (accurate observation and representation) 
versus conceptual thinking 

■ understanding and organising ideas 

■ reasoning 

■ understanding causal relationships 

■ planned systematic enquiry 

■ problem solving 



Although this study was small in scale, the categories 
listed above proved helpful in confirming and extending 
the basic structure developed by the project team. 

We were finally able to create a restructured version 
of the six-level model, this time including sub-categories 
of basic thinking skills (which we renamed building 
understanding) and productive thinking as shown 
above in Figure 5. This involved a number of small 
adjustments. We split concrete versus conceptual 
thinking into its two components and substituted 
caring, value-grounded thinking forthe original reflective 
thinking. Value-grounded thinking is really about 
what makes forgood thinking, and includes the use 
of metacognition. Some otherminoradjustments 
were made to the wording and, following Hannah and 
Michaelis (1977), we adopted the term information 
gathering to subsume skills involved in perception, 
recognition and knowledge retrieval. 



■ creating 

■ value-grounded thinking (orcaring thinking). 



5 

This term is not meant to imply that values and beliefs are rigid or static. 





All the components of the six-level structure shown 
in Table 7 are retained in our revised integrated 
framework. The new framework has the advantage 
of containing rather more detail, but at the same 
time having a simplerstructure, with onlyfour main 
categories and two levels. In essence, it is made up 
of the three cognitive components which we identified 
in Bloom’s taxonomy (1956) plus a superordinate 
self-regulatory/ metacognitive level. This does not 
mean that it is restricted to the cognitive domain, 
since it is intended to accommodate Lipman's critical, 
creative and caring thinking (1995). The two levels 
illustrated in Figure 5 represent the distinction between 
cognitive skills and strategic and reflective thinking. 

We decided to use the terms strategic and reflective 
because they are accessible and relatively 
uncontentious. We also use the terms engagement 
and value-grounded to convey the idea that we are 
interested in the conative and affective aspects 
of thinking, and see our framework as applying to all 
kinds of thinking, including the 'emotional intelligence’ 
areas which Gardner (1983) describes as interpersonal 
and intrapersonal intelligence. 

In the cognitive skill part of the framework, the 
three components (information gathering, building 
understanding and productive thinking) are ordered 
from left to right, but this is not meant to imply that 
all thinking processes include the middle level of 
building understanding, as it is possible to go straight 
from information gathering to productive thinking. 
Information gathering is a prerequisite foreither 
building understanding orproductive thinking, but it 
is not necessarilya simpleror less conscious process. 
Although it very often happens that thinking develops 
through distinguishable (if overlapping) phases, from 
information gathering to building understanding to 
a sound judgement ordeeper understanding, this is 
not always the case, since these phases can take place 
in parallel or in complex systems with movement in both 
directions (as when it is found at a late stage of problem 
solving that a vital piece of information is missing). 



The structure of our integrated model brings out 
the fact that strategic and reflective thinking can 
be used in conjunction with information gathering, 
building understanding orproductive thinking. We can, 
forexample, decide to engage in the task of trying 
to retrieve early memories, do it carefully in order to 
minimise memory distortion, and monitorand manage 
the process through reflection and by checking accuracy 
against external criteria. Equally, we can reflect on 
our understanding of how emotions are expressed, 
move on to develop a more productive understanding 
of emotions (people get angry when... ) and develop 
more effective strategies for managing emotions 
when we interact with others. We believe that the 
two-level structure of the model is a more accurate 
representation of how people think than a multilevel 
hierarchy. It also easily accommodates the various ways 
in which young and novice learners think strategically 
and reflectively as they develop information-gathering 
skills and build understanding. 

The relationship between the two levels of strategic 
and reflective thinking and cognitive skills is, 
nonetheless, hierarchical in the sense that strategic 
and metacognitive thinking cannot take place unless 
there is or has been information gathering, building 
understanding orproductive thinking going on. There 
is anotherdifference between the two levels, in terms 
of the nature and qualityof experience involved. 
Cognitive skills are procedures which can become 
automatised and are not necessarily associated with 
effort or emotion. However, strategic and reflective 
thinking are always highly conscious and are often 
experienced as involving will and/ or emotion as well 
as cognition. 

The two-way arrow between strategic and reflective 
thinking and cognitive skills does not fully represent the 
possible relationships between them. In many thinking 
and learning situations there certainly is two-way 
interaction, often in such a dynamic mannerthat the 
experience is a holistic one. However this does not 
always apply, since cognitive skills can be exercised in 
non-strategic and unreflective ways. On the other hand, 
it is impossible to operate at the level of strategic, 
value-grounded thinking without knowledge access and 
othercognitive skills coming into play. It is important 
to note that we are not making any claims about how 
thinking starts orabout causality. The impetus for 
strategic or reflective thought may be situationally 
specific, as when a particularproblem causes cognitive 
conflict, or it may flow from a well-established 
disposition or 'habit of mind'. What we do claim is that 
when thinking is strategic and reflective (involving the 
exercise of conscious purpose and a carefully executed 
plan), meaningful learning - to use Ausubel's phrase 
(1968) - is more likelyto occur. 
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Strategic and reflective thinking are not easy, since 
they require sustained concentration, not only on the 
matter in hand, but also on how a task is conceived 
and whether or not there should be a change of strategy 
in the light of new and previous experience. Strategic 
and reflective thinking may involve considering the 
meaning of an activity in holistic as well as analytic 
ways. This kind of thinking is important when embarking 
on activities which make considerable demands 
on a person, such as an academic orvocational 
course orproject. It can also be extremely valuable 
in dealing with much smallerissues; forexample, 
when there is a challenge to an assumption orbelief, 
ora communication problem. Most significantly, it is 
often what changes what could be a routine process 
into a learning experience. The development of strategic 
and reflective thinking is acknowledged to be a major 
goal of higher education. We see itas equally important 
in lifelong learning. 

The integrated framework proposed here is in some 
respects similarto the map of the thinking domain 
created by Swartz and Parks (1994). However, Swartz 
and Parks do not deal with information gathering and 
use rather more categories to cover what we have 
called building understanding and productive thinking. 
Although they constantly stress the importance of 
metacognition, Swartz and Parks do not represent it 
on their map. Our new integrated framework is not only 
simple in structure, but is compatible with the National 
Curriculum categories as well as with leading theories 
about thinking and learning. The motivational and 
regulatory aspects of thinking (which cognitive 
psychologists think of as functions involving the 
'central executive’) are distinguished from cognitive 
skills; but, unlike Marzano, we do not see the need 
to distinguish between a self system and metacognitive 
system. As argued in Section 3.2.2 of this report, 
conscious planning, monitoring and evaluating 
functions are not neatly separable into two components, 
as presented in Marzano’s model. This is recognised 
by Pintrich, who includes Marzano's self-system and 
metacognitive-system functions within a unified 
framework of self-regulated learning. 



4.4 

Which frameworks have the most to offer for 
general use with post-16 learners? 

By 'general use', we mean 'widespread application as 
a wayof giving purpose and structure to the experience 
of teaching and learning'. What is needed are accessible 
accounts of thinking and learning which are compatible 
with the integrated framework shown in Figure 5. 

We have notfound a single authorwho meets this 
requirement, although atan earlierstage of our 
research we thought that we might be able to 
recommend Marzano’s work above all others. His 
framework has two main advantages. It is built on 
psychological theory, and has been used to classify 
the outcomes of educational interventions in the largest 
meta-analysis of its kind ever undertaken (Marzano 
1998). However as explained in ourevaluation 
(Section 3.2.2), there are certain problems with his 
approach. First, we do not believe that it is helpful 
to distinguish between the self system and the 
metacognitive system. Second, Marzano’s set of 
knowledge utilisation categories omits reasoning 
and creative thinking. Third, in his first three cognitive 
categories, he defines some terms in ways which 
diverge from common usage and from the well-known 
meanings in Bloom's taxonomy (1956) and its later 
revision by Anderson and Krathwohl (2001). 

Forthese reasons, we decided that three 
complementaryframeworks are needed ratherthan 
one. Pintrich’s framework of self-regulated learning 
(described and evaluated in Section 3.5.2) best conveys 
the meaning of what we have called strategic and 
reflective thinking, using a variation of the familiar 
'plan-do-review' cycle. Halpern (see Section 3.4.2) 
provides a popular productive thinking framework, 
accompanied by resource materials designed for 
post-16 learners. Anderson and Krathwohl’s updated 
and extended revision of Bloom’s taxonomy (see 
Section 3.3.2), which can be used with anyage and 
ability group, provides a vocabularyfordescribing 
specific knowledge and skill objectives. It covers basic 
thinking skills as well as single processes, which, 
especially when combined, constitute productive 
thinking. Readers who wish to considerpoints forand 
against each of these three frameworks are referred 
to the graphically structured arguments in Appendix 3. 



LSRC reference 



page 48/ 49 



Section 5 



Some ideas for using the frameworks 



5.1 

Using a five -category framework to classify key 
skills objectives 

If a framework of thinking skills is to be of practical 
value in preparing and communicating curriculum 
objectives, users must be able to apply it consistently. 

As a first step towards evaluating the applicability 
of the newframework presented in Figure 5 (page 45), 
we used a five-category version of it to classify Key 
Skills knowledge objectives, Levels 1-4 6 . We did this 
because this widely used national set of objectives 
incorporates generic thinking and learning skills, 
especially in whatare known as the 'wider key skill' 
areas of Working with Others, Improving Own Learning 
and Performance and Problem Solving. We were 
interested not only in whetherthe new framework could 
be applied consistently, but also in what ouranalysis 
would tell us about the importance given to different 
types of skill at different levels. The thinking skill 
categories that we used to classify the objectives 
are shown in Table 8. 

Having decided on the five categories and theirdefining 
characteristics, the research team undertook an 
analysis of the relevant key skills documents. Ourfirst 
aim was realised in that full agreement was reached 
about the categorisation of all knowledge objectives 
across the six key skills areas. It was possible to resolve 
the small number of differences of opinion by referring 
backto the definitions in Table 8. 

Figure 6 shows the relative emphasis given to five 
types of thinking across all six key skills areas. 

Building understanding (39%) and productive thinking 
(42%) are predominant. Figure 7 shows the proportional 
emphasis within each level and reflects the more 
demanding nature of successive levels. It is noteworthy 
that only at Key Skills Level 4 is there a significant need 
forreflective thinking and forthe strategic management 
of thinking which it is reasonable to expect of 
independent learners. 



Figure 8 charts the percentage of each type of thinking 
found within each of the key skills areas. It is apparent 
from the chart that Communication, Application 
of Numberand Information Technology contain 
a higher percentage of building understanding skills 
than Working with Others, Improving Own Learning 
and Performance and Problem Solving. The first three 
also contain lower percentages of reflective thinking 
and strategic management of thinking. 

In the classification exercise reported above, 
we found it feasible to make the broad distinction 
between building understanding and productive 
thinking. If we had attempted to make distinctions 
between sub-categories, we should have had much 
more difficulty. This is because the sub-categories 
inevitably overlap, and because what is problem solving 
forone person maybe creative thinking foranother. 
Nevertheless, we believe that the simplified version 
of ourframework has potential as a tool for use in 
planning and evaluating courses and curricula, and 
constructing and grading of assessment tasks. We give 
examples below to illustrate how ourfive categories 
were able to accommodate all the key skills objectives. 

Information gathering: identifythe person you will 
see to review your progress and where and when this 
will take place. 

Building understanding: make changes suggested 
byyoursupervisor. 

Productive thinking: seek and actively use feedback 
and support from relevant sources to help you to 
meet targets. 

Reflective thinking: monitorand critically reflect on what 
you are learning and howyou are learning, noting the 
choices you make and judging theireffectiveness. 

Strategic management of thinking: adapt yourstrategy 
to overcome difficulties and produce the quality of 
outcomes required. 



Table 8 

Five types of thinking 
skill and their definitions 



Category of thinking 


Characteristics 


Information gathering 


Obtaining information from memory, 
or by observing, listening or reading 


Building understanding 


Understanding situations and carrying 
out simple procedures 


Productive thinking 


Thinking which leads to deeper 
understanding, orto a judgement, 
solution or product 


Reflective thinking 


Monitoring or reviewing one's thinking 
to improve performance 


Strategic management of thinking 


Thinking that leads to a change in plans 
or approach to a problem 



6 
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Figure 6 

Percentage distribution 
of five types of thinking 
across key s kills 

■ Strategic management 
of thinking 5% 

■ Reflective thinking 10% 

■ Productive thinking 42% 

■ Building understanding 39% 
Information gathering 4% 




Figure 7 

Percentage distribution 
of five types of thinking 
within each keyskill level 
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Figure 8 

Percentage distribution 
of five types of thinking 
within each keyskill 
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5.2 

Other ways of using thinking skill frameworks 

Everyone who is involved in lifelong learning needs 
to have some understanding of its nature and purpose. 

A framework for understanding thinking and learning 
can be used at any level of generality; forexample, 
as a guide to the formulation of a mission statement 
orin formulating specific learning objectives and 
assessment items. When a theoretical framework 
is used consistently and explicitly, it is likely that 
communication within an educational ortraining context 
will be enhanced, as well as communication with the 
outside world. 



There are a numberof subject disciplines which have 
as theirfocus the study of human beings. These include 
philosophy, psychology, sociologyand anthropology, 
where almost every aspect of human behaviour is 
of potential interest. Geographers and historians 
are clearly interested in a broad spectrum of human 
behaviourand we could add otherdisciplines to the list. 
The point is that in the humanities, just as much as in 
the sciences, there are benefits to be obtained through 
collaboration and this too requires a shared language 
about how people think and learn. It is certainly 
possible fora thinking skills framework to be drawn 
up for each subject area, but if this were done, the 
differences would probably lie only in the detail. In our 
view, many benefits would flow from the interdisciplinary 
development of a common framework, especially 
if care were taken to avoid the use of the kind of esoteric 
language which tends to maintain artificial boundaries 
between traditional academic subjects. 




Understanding thinking and learning is important not 
only in academic study, but also in professional and 
vocational courses. There is a strong case to be made 
forthe development in the UK of post-16 qualifications 
which incorporate some of the positive features 
of the baccalaureate qualification, whether modelled 
on French, European or International exemplars. These 
have traditionally included the philosophical study of 
theories of knowledge, but have not included the study 
of theories of learning. However, it would make good 
sense for thinking and learning to form the core of such 
studies, associated with another subject of choice in 
which human behaviour is the focus. An understanding 
of thinking and learning frameworks should inform 
the planning of appropriate curricula. 

There are many ways in which the use of thinking 
skills frameworks can be built into teachertraining 
and professional development. Teachers need regular 
opportunities to reflect on theirown learning and 
style of teaching. Without such opportunities, they are 
ill-prepared to engage learners in similardiscussions. 
Thinking skills frameworks can help to provide the 
necessary lexicon of thinking and learning, to develop 
a common language applicable in every subject area. 

Elsewhere in this report we have referred to the 
use of thinking skills frameworks in the planning 
of instruction, in teaching, in assessment, and in the 
alignment of ail three. Planning can be done at several 
different levels, but it is the teacher who has, on the 
basis of formative and summative assessment, 
to make constant adjustments while teaching in order 
to facilitate learning. The skills required cannot be 
learned from textbooks alone, but are undoubtedly 
capable of development and fine-tuning. The teacher’s 
job is to ensure that learning takes place, and as 
teachers develop expertise, their constructs about 
teaching and learning become more sophisticated. 

Being able to discuss those constructs within 
a community of practice and in relation to theory-based 
frameworks makes a teacher not only a learner, 
but a practitioner-researcher. 

Thinking skills frameworks inform a numberof reputable 
initiatives in educational and workplace settings. 

Among these are Feuerstein’s (1980) Instrumental 
Enrichment (IE) programme, Thinking Skills at Work 
(Blagg, Lewis and Ballinger 1993), Teaching Thinking 
Skills in Vocational Education and Training (Soden 
1993) and approaches based on Lipman’s community 
of enquiry (Lipman, Sharp and Oscanyon 1980). There 
are many other initiatives which would benefitfrom 
being explicitly grounded in a theory-based framework of 
thinking skills; for example, de Bono’s (1976) strategies 
forteaching thinking and Lake and Needham's Top Ten 
Thinking Tactics (1993). 



It is not justteachers, but learners who need 
to develop a mature understanding of thinking 
and learning, especially in contexts where learners 
have to take a large share of the responsibility for 
theirown progress. Theyneed to consider not only 
theirimmediate needs, but the possible value 
of 'transferable skills’ (the thinking that underlies 
key skills initiatives). Good Communication, effective 
Working with Others and a commitment to Improving 
One’s Own Learning and Performance are clearly 
valuable qualities. They can be developed in many 
ways, not justthrough formal education and training. 

As Lipman (1991) argues, it is farfrom clearthat 
the best way to develop an intrinsic interest in thinking 
and learning is through prescribed activities. Lipman 
advocates the communityof enquiry approach to 
thinking and learning, through philosophical discussion 
of issues of concern to a group. Yet even here, 
participants need to understand the theoretical 
framework within which the community operates. 

Another use of a framework for understanding 
thinking and learning is as a research and evaluation 
tool. Forexample, Vermunt and Verloop (1999) write 
persuasively about congruence and friction between 
learning and teaching. They suggest that 'congruence' 
exists when (a) teacherdirection is high and student 
self-regulation is low; and (b) student self-regulation 
is high and teacherdirection is low. Atheory-based 
framework of approaches to learning like the one 
produced by Vermunt (1996) can lead to new ways 
of assessing psychological and pedagogical aspects 
of learning environments. It then becomes possible 
to relate what are presumed to be indicators of quality 
to outcome measures. There is a serious lack of 
evidence-based information of this kind, without 
which inspection regimes lack credibility. 

5.3 

Which frameworks are best suited to specific 
applications? 

In Section 4.4, we gave reasons for recommending 
three frameworks for general use. Two of these 
(Halpern; Anderson and Krathwohl 2001) are written 
in an accessible style, although only Halpern writes 
explicitly with post-16 applications in mind. There 
is continuity between Bloom (1956) and Anderson and 
Krathwohl (2001) in that David Krathwohl was a member 
of Bloom’s original team. In fact, teachers can still learn 
a lot from Bloom’s famous book, which was written with 
college students in mind and shows few signs of its 
age. It has the advantage of being very concise and 
very well written. Pintrich's framework of self-regulated 
learning can be understood in tabularform, but does 
need a more accessible exposition. 

These three frameworks are, however, not the only 
ones that can help teachers acquire what in Section 5.2 
we called a 'lexicon of thinking and learning'. An 
outstanding resource is the latest edition of Costa’s 
Developing minds (2001), which contains chapters 
by Baron, Ennis, Marzano, McTighe, Paul, Perkins, 
Presseisen and Tishman, among others. 
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We believe also that the following frameworks 
summarised in Appendix2 can help the reader to 
understand the general subject of thinking and learning 
(which is not to say that all the original sources are 
accessible): Baron, Carroll, Gardner, Guilford, Hannah 
and Michaelis, Jewell, Koplowitz, Lipman, Quellmalz, 
Presseisen, Romiszowski, Sternberg, Vermuntand 
Verloop, and Williams. The work of Gardner Presseisen, 
Sternberg and Vermunt is readily available and 
particularly relevant for teacher educators. 

If the subject of thinking skills, especially critical 
thinking, is approached from a philosophical 
perspective, the frameworks of Ennis, Lipman 
and Paul can be usefully compared and contrasted. 

An in-depth psychological perspective on critical 
thinking is provided byHalpern. As Sternberg's earlier 
work on critical thinking is available onlyon microfiche 
(1986), we recommend the latest edition of his 
general textbook on cognitive psychology (2003). 

For understanding developmental perspectives 
on critical thinking, King and Kitchener’s (1994) book 
is the best source, with Perryand Belenky (1986) 
providing both historical and theoretical context. 

For instructional design purposes, we have 
highlighted Anderson and Krathwohl’s revision (2001) 
of Bloom’s taxonomy (1956). Readers who want 
a comprehensive treatment of instructional design 
should consultj onassen, Tessmerand Hannum (1999). 
In addition, we suggestthatconsideration be given 
to the frameworks developed by Romiszowski (1981), 
Hannah and Michaelis (1977) and Hauenstein (1998). 
These all deal with psychomotor and affective learning 
objective (as well as cognitive ones) and are concerned 
with stages in the development of understanding as 
well as with the end result. If the focus of instructional 
design is creative thinking, the most useful frameworks 
are those of Gouge and Yates (2002) and Williams 
(1970), while the work of Halpern and Lipman 
is also relevant. 

Of all the authors we have reviewed, Marzano is 
noteworthy fortranslating research findings into 
teaching recommendations, framing these within the 
structure of his new taxonomy (Marzano 1998, 2001a). 
His is a pragmatic approach to pedagogy. In contrast 
with Lipman and Paul, neither Marzano nor Anderson 
and Krathwohl seem desirous of radically transforming 
educational practice. 

A wide spectrum of views about what constitutes 
effective pedagogy is represented in the work we have 
evaluated. It would be unwise to assume that thinking 
skills can onlybe developed in particular kinds of 
learning environment that are advocated by certain 
theorists orfashionable atthe time. However, it does 
seem clearthat if strategic and reflective thinking 
are to be developed, learners need to be in situations 
where they have opportunities to use that kind 
of thinking. Theorists usually have preferences about 
ways of encouraging good thinking, meaningful learning 
and deep or strategic approaches to study. All agree 
that learning and thinking are active processes in which 
new connections are made. 



Vermunt and Verloop (1999) place approaches 
to learning and instruction on a continuum between 
teacher-regulated and student-regulated, and point 
out that there may be differences between pedagogic 
practice and beliefs and student conceptions of 
learning and abilityto regulate it. The continuum 
of theoretical positions in the work we have reviewed 
ranges from Gagne (1965, 1985) and Ausubel at the 
teacher-regulated end to those like Feuerstein, Hannah 
and Michaelis (1977) and Romiszowski (1981), who 
favourforms of 'guided discovery’, to those like Lipman 
and Paul who advocate learner-centred approaches. 
Learner empowerment through the social construction 
of knowledge is strongly valued by Belenky et al. (1986), 
King and Kitchener (1994), and J onassen and Tessmer 
(1996/ 97). Gardnerand Vermunt believe in the 
importance of tailoring instruction to meet group 
and individual needs. 

Vermunt, in common with very many othertheorists, 
emphasises process over content in much of his 
writing, arguing that self-regulation is the key to lifelong 
learning ratherthan the accumulation of knowledge. 

The onlytheoristto deal adequately with whatteachers 
and learners can do to improve the acquisition and 
retention of knowledge and skills is Halpern. We believe 
this to be a neglected but still very important area. 

In ourGateshead College study (Moseley 2003), 
we found that teachers rated the internalisation 
of knowledge and skills as being a majoraim, next 
in importance to the most important of all, the 
development of value-grounded thinking. 

In the actual practice of teaching, it is possible 
fora teacherto keep a simple framework in mind, 
as a means of monitoring the kind of thinking expected 
of students. This is especially important in the process 
of questioning and when discussing a topic with a class, 
group or individual. The simple four-category system 
developed in this project (see Figure 5; Section 4.3) is 
very suitable forthis purpose. Anderson and Krathwohl’s 
(2001) six process categories (remember, understand, 
apply, analyse, evaluate, create) can also be used in 
this way. Manyteachers find theycan readily internalise 
Gardner’s seven (now nine) kinds of intelligence 
(see Appendix2) in order to monitorlearning activities 
in those terms. The ’3Cs’ of critical, creative, caring 
thinking, which are derived from Lipman and used by 
J ewell (1996) are also easily memorised and applied. 

Our brief description and evaluative summaryof 
Alts huller's theory of inventive problem solving (TRIZ) 
(1996, 1999, 2000) does not do it justice. It is now 
taught in a numberof universities in the UK and 
has been widelytaken up in many countries where 
technological innovation is valued. It has the unique 
qualityof organising creative thinking. Although coming 
up with inventive solutions to practical problems still 
depends on analogical thinking and looking for patterns, 
the task is much simplified by applying Alts huller's 
'algorithmic' procedures. These are the result of many 
years of systematic data gathering and his analysis 
of existing patented solutions. 



Consultancy (whether in educational or business 
contexts) is anotherarea in which frameworks of 
thinking and learning are widely used. We have not 
attempted to identify all the frameworks that inform 
practice in this field, many of which are models of 
learning styles rather than thinking skills. Others 
(eg Senge 1990) are simply sets of problem-solving 
heuristics. We mention here three particular 
frameworks. One of the early rational problem-solving 
approaches was the Kepner-Tregoe (Kepnerand Tregoe 
1965) tenets of effective decision making - rational 
thinking. Koplowitz (1987) has applied his theory of 
stages in adult cognitive development in business 
settings and argues that better decisions are made 
when people move beyond logical analysis to more 
systemic and holistic ways of thinking. Although it was 
not devised with consultancy applications in mind, 
Vermunt and Verloop’s categorisation of learning 
activities (1999) appears to us to be highly applicable 
as a way of understanding how learning develops 
(or not) in any organisation. 

Assessment is another major area in which 
thinking skill frameworks, especially those dealing 
with educational objectives, are extremely relevant. 

As Ennis recognises, the assessment of critical thinking 
is a problematic area, despite being one in which he 
is personally involved as an author (Ennis and Millman 
1985; Ennis and Weir 1985). We believe, however, 
that the most useful framework for assessing extended 
written assignments at post-16 level is the SOLO 
taxonomy of Biggs and Collis (1982). This has the merit 
of being easily communicable to students. Examples 
of relevant work in the appropriate subject area can be 
presented to students to illustrate each of the five SOLO 
levels. Students can also assess such pieces of work 
(including theirown). 

When it comes to the assessment of personal 
qualities and dispositions, especia Ilyas displayed 
in group situations, further problems arise. However, 
these are not necessarily insuperable, as experience 
with the wider key skills portfolios suggests (LSDA 
2002). The lists of dispositions produced by the 
following authors maybe helpful in this context: 

Baron (1985), Ennis, Halpern, J ewell (1996) and 
Paul. Costa and Kallick’s (2000a, 2000b) 16 'habits 
of mind’ 7 and the seven dispositions putforward 
by Perkins, J ay and Tishman (1993) as the basis 
of theirdispositional theory of thinking are also worth 
considering. Costa, Kallick and Perkins (2000) address 
the topic of assessing and reporting on 'habits of mind'. 8 



7 

A phrase used byDewey(1930). 

8 

Fora categorisation of Costa and Ka Nick's 'habits of mind' using our 
integrated framework, see Appendix4. 



Thinking skill frameworks are also valuable in 
research and evaluation. Pintrich is a good example 
of a researcher who has developed ways of assessing 
learning, aided by his theoretical framework of 
self-regulated learning. Sternberg is another, with 
his triarchic theory of successful intelligence and his 
claim that 'triarchic teaching' is more effective than 
traditional approaches (2002). Vermunt and Verloop’s 
(1999) categorisation of learning activities and 
Vermunt's broader framework for understanding 
approaches to learning (1996) have led them and 
other researchers to find ways of assessing 
psychological and pedagogical aspects of learning 
environments. Feuerstein's Instrumental Enrichment 
(IE) intervention programme (1980) is typically 
evaluated using closely related cognitive measures 
from the Learning Potential Assessment Device (Hattie, 
Biggs and Purdie 1996; Romneyand Samuels 2001). 

In fact, ail thinking skill frameworks can be used to 
generate research questions. 

Finally, meta-analysis can be structured by using 
categories from thinking skill frameworks. This makes 
it possible to compare the effect sizes produced by 
different types of educational intervention. It was in 
this waythat Hattie, Biggs and Purdie (1996) were able 
to evaluate the effects of learning skills interventions 
on student learning (including 14 studies with adult 
students). Theydid so byusingthe SOLO taxonomy 
(Biggs and Collis 1982) to categorise interventions 
as being uni-structural, multi-structural, relational or 
extended abstract. Theyalso compared 'near transfer’ 
with 'fartransfer' effects. Further information and 
a summaryof the results are provided in Section 6.2. 

We believe that future meta-analyses should build 
on this kind of approach. 

Marzano’s (1998) meta-analysis was broader in scope 
and largerin scale than anyotherwe have found. 
Marzano categorised all studies, including 147 studies 
at college level, using the categories and sub-categories 
from his new taxonomyof educational objectives. 
Forfurther information, see Section 6.2. Marzano’s 
overall conclusion (1998, 135) was as follows: 

The effective teacher is one who has clear instructional 
goals. These goals are communicated both to students 
and to parents. Ideally, the instructional goals address 
elements of the knowledge domains as well as the 
cognitive, metacognitive, and self-system. Even if the 
instructional goals focus on the knowledge domains 
only (as is frequently the case in public education), 
the teacherstill uses instructional techniques that 
employthe cognitive system, the metacognitive 
system, and the self-system. Perhaps, above all, the 
teacher understands the interrelationships among 
the knowledge domains, the cognitive system, the 
metacognitive system, and the self-system, and uses 
that understanding to make the myriad of instructional 
decisions thatoccurin a single lesson. 
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Section 6 



Summary of what is known and the need for further research 



6.1 

Introductory outline 

In this section, we summarise evidence from over 
4000 empirical studies to show that thinking skills 
approaches can be very effective, especially 
those targeted at the skills of metacognition and 
self-regulation. We also present a list of research 
findings about thinking and learning which we believe 
to be based on extensive and sound empirical research. 

Turning to matters of theory, we note a degree 
of rapprochement between cognitive, constructivist 
and some recent behaviourist formulations. However, 
we do not believe that it has been established that 
meaningful learning can take place only when there 
is a low level of teacher direction. 

We pointoutthata greatdeal of educational practice 
is based on sets of widely accepted but usually 
untested beliefs, values and assumptions. Empirical 
evidence leads us to question one of these 
assumptions: the idea that to be effective, teachers 
need to base their practice on coherent theory. 

After listing a large numberof areas where it cannot 
be claimed that either beliefs or practice have strong 
research support, we prioritise certain areas where 
further research is needed in the post-16 sector. 

We conclude by offering suggestions for policy-makers 
based on our understanding of the literature. 

6.2 

I n which areas is there extensive or widely 
accepted knowledge? 

Theorists and taxonomists who categorise thinking 
skills do so on the assumption thatthe terms they use 
have meaning: in other words, thatthere are at least 
some skills, abilities ordispositions which are 
recognisable in different contexts. The question as 
to how far people are able to make use of those skills, 
abilities and dispositions in new situations, especially 
when learning is required in addition to preparedness 
and recall, is one requiring an answer based on 
experience, not theory. In our view, empirical research 
has amply confirmed that thinking and learning skills 
can be taught in such a way that the skills can 
successfully be applied in different (albeit usually 
closely related) areas. 

While it was not in our remit to evaluate the evidence 
as to the benefits of teaching thinking skills, we refer 
here to two relevant meta-analyses. In each of these, 
a thinking skill framework was used to categorise 
the results; and in both cases, it was found that 
interventions directed at metacognitive thinking skills 
were highly effective. 



Hattie, Biggs and Purdie (1996) confined their 
interestto 271 effectsizes in 51 'learning skills 
interventions’. Theyfound that 'unistructura I' 
approaches (interventions directed at single-skill 
outcomes) were the most effective, with a large 
mean effect size of 0.83. Among the most effective 
unistructural interventions were those addressing 
memory and reproductive performance. However, 
'relational interventions’ with 'neartransfer' were 
also highly effective, the mean effect size being 
0.77. The authors say (1996, 105) that relational 
interventions 'are integrated to suit the individual’s 
self-assessment, are orchestrated to the demands 
of the particulartask and context, and are self- 
regulated with discretion.' Relational interventions 
frequently have a metacognitive emphasis and include 
attributional retraining studies (which had a mean effect 
size of 1.05). Using 'structural aids’ (with a strategy 
emphasis) was also moderately effective, the mean 
effect size being 0.58. Overall, in line with the weight 
of previous research, transfer effects were larger with 
'near' than with 'far transfer’. Hattie, Biggs and Purdie 
also found thatthe mean effect size for self-directed 
interventions (0.70) was higher than forteacher-directed 
interventions (mean effect size 0.44). The number 
of studies involving adults outside universities was 
too small to permit any breakdown, but the results 
were not encouraging, the mean effect size being small 
(only 0.22). 

Marzano (1998), found that college students responded 
just as well as school pupils when data from more than 
4000 studies were aggregated. He confirmed Hattie, 
Biggs and Purdie's finding (1996) that techniques 
designed to be used by students led to significantly 
better results than those designed to be used by 
teachers. Although there was enormous diversity in 
the intervention studies selected by Marzano, ranging 
from a focus on specific skills (such as memorisation) 
to the use of disposition-monitoring strategies, he made 
the following claim (1998, 127) about the importance 
of metacognition: 

... instructional techniques that employed the 
metacognitive system had strong effects whether they 
were intended to enhance the knowledge domains, the 
mental process within the cognitive system, the beliefs 
and processes within the self-system, orthe processes 
within the metacognitive system itself. 

Overall, Marzano found that interventions which engage 
either the self system orthe metacognitive system lead 
to better knowledge outcomes (bysixand five percentile 
points respectively) than those which are directed 
onlyatthe use of cognitive skills. Nevertheless, 
there are some types of very effective intervention 
at the cognitive skill level. These are interventions 
which address: experimental enquiry, using analogies, 
comparing and contrasting, idea representation and 
the storage and retrieval of knowledge. 




We can summarise these meta-analyses by saying 
that there is powerful empirical evidence that 
thinking skill interventions can be very effective 
at all levels, but especially if they are directed 
at metacognition, self-regulation and what we have 
called 'value-grounded thinking’. Their effectiveness 
is likelyto be greater if they are used forlearner 
self-regulation ratherthan coming fully under teacher 
control. However, well-focused interventions at the 
cognitive level can also be very effective. These include 
interventions with a focus on experimental enquiry 
and idea representation, as well as approaches 
to study support such as using cues and questions 
to aid retrieval. Unfortunately, we cannot be confident 
that substantial learning gains are achievable with 
post-16 learners not in higher education, as there 
are so few published studies. 

Apart from the evidence coming from the meta-analysis 
of high-quality empirical research, the following 
statements can be backed up by research findings 
(although, again, much of the evidence comes from 
younger age -groups). 

■ Learners have different preferences, needs, 
backgrounds and skills. 

■ Most learners think of teaching as the transmission 
of knowledge. 

■ Many learners have had negative experiences at school. 

■ Thinking and learning are active processes. 

■ Learning often involves a re-conceptualisation of 
information. 

■ Motivation is the keyto active learning. 

■ Students need to have appropriate, specific and 
challenging goals. 

■ The quantity and qualityof instruction make 
a big difference. 

■ Learners can benefit from thinking skills interventions 
irrespective of age. 

■ Learners can benefit from thinking skills interventions 
irrespective of ability. 

■ Teaching for critical, creative and caring thinking is 
rare in FE colleges. 

■ Manyteachers and employers want compliantthinkers 
and learners. 

■ There are all-round benefits from learning in groups 
of 15 or less. 

■ Learning is affected by contextual factors. 

■ Progress is facilitated by constructive formative 
assessment. 



Apart from the research evidence presented above, 
there are a numberof areas in which there is a high 
level of consensus about how people think and 
learn. We have alreadydrawn attention to structural 
similarities in many of the frameworks evaluated in 
this report, and shown how it is possible to construct 
an integrated framework within which several of them 
can be accommodated. This is a framework in which 
there is constant interplay between cognition, feelings 
and motivation. The assumption is made that there 
is a conscious self which sets goals and is motivated 
to achieve them; yet is, at the same time, strongly 
influenced by unconscious and interpersonal dynamics. 

There is wide acceptance among psychologists and 
educators of the idea thatthinking in individuals 
and groups is shaped through interpersonal interaction. 
Cognitive psychology is compatible with constructivist 
conceptions of learning and with the importance 
of person-situation interactions. As we have seen, 
theorists such as Pintrich and Vermunt see contextual 
factors as being highly important in relation to 
self-regulated learning. The socio-cultural contexts 
in which learning takes place are also generally 
considered to exert powerful influences, even within 
the behavioural tradition. Forexample, Strand, 
Barnes-Holmes and Barnes-Holmes (2003, 105) 
suggest 'that advances in our understanding 
of choice behaviorand verbal behaviorput us within 
reach of a comprehensive framework for making 
sense of the interconnectedness of social, self, 
and academic development.' 

The widely held view that constructivist beliefs 
aboutthinking and learning are incompatible with 
teacher-directed or behavioural approaches to 
instruction is an exaggerated position which has 
only a modest level of support from the meta-analytic 
findings which favour self-regulation over teacher 
direction. The fact is that teacher-directed approaches 
can also be effective in teaching thinking. Strand, 
Barnes-Holmes and Barnes-Holmes (2003) refer 
to several examples of this within the behaviourist 
paradigm. As Ausubel argued, meaningful learning 
can occurwith varying degrees of direction by teachers. 

A great deal of educational policy and practice is based 
on sets of widely accepted but usually untested beliefs, 
values and assumptions. We now list some of these, 
without embarking on a radical critique of power 
relationships and political ideologies. Manypeople 
involved in post-16 education and training would agree 
with the following statements. 
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■ Most people are potentially more competent than 
they think. 

■ Learners need to feel valued in personal, social and 
cultural terms. 

■ Learners need to see instruction as being personally 
relevant. 

■ Learners need to understand why theirteachers 
do whatthey do. 

■ Thinking and learning are facilitated by social 
interaction and discussion. 

■ Good teaching helps learners to move from dependency 
to independence. 

■ Good thinking and learning help to develop valuable 
personal qualities. 

It is important to acknowledge that while beliefs 
such as these have a commonsense appeal orcan 
be justified on moral grounds, few are grounded in 
empirical research. 

We now need to question a majorassumption made 
at the start of the present project. Our initial brief was 
to evaluate thinking skill taxonomies, as it was thought 
that principled or theory-grounded classification 
systems are preferable as a basis for practice than 
simple lists orsets of heuristics. We subsequently 
decided to evaluate theory-grounded frameworks as 
well as taxonomies, but still made the assumption that 
thinking skills approaches derived from theoretically 
coherent frameworks, such as those based on the work 
of Feuerstein and Lipman, are more effective than those 
which we had to exclude, such as the subjects covered 
in de Bono’s Cognitive Research Trust (CoRT) Thinking 
Lessons (1988). 

Hattie, Biggs and Purdie (1996) have shown thatthis 
ratherfundamental assumption maybe ill founded. In 
their meta-analysis of learning skills interventions, they 
found a slightly higher mean effect size for 'atheoretica I' 
interventions (n =36 effects) than fortheory-based 
interventions (n =234 effects). Although the difference 
was not statisticallysignificant, itappears thatit makes 
no difference to practitioners whethera change in their 
normal practice has, ordoes not have, an academically 
respectable theory or well-structured taxonomy 
to support it. 

To understand and explain this state of affairs, 
researchers and teachers will need to considerthe 
following possibilities. 



1 

Many theories aboutthinking and learning are 
unimportant. 

2 

It is often difficult to work out how theories can be 
effectively applied. 

3 

The communication of relevant theory to practitioners 
is often poor. 

4 

Theories are often too complex ortoo abstract for 
classroom application. 

5 

Teachers rarelymodifytheir'theories in use’. 

6 

Many teachers are not interested in theoryordo not 
understand it. 

Although the lack of evidence in favour of theory-based 
educational interventions is not widely appreciated, 
we have mentioned it here as being based on 
extensive research and being consistent with our 
own observations that policy-makers and teachers 
are more often influenced by pragmatic than by 
theory-based concerns. 

6.3 

In which areas is knowledge very limited or highly 
contested? 

In our evaluations we have commented on the explicit 
and implicit value systems communicated by each 
theorist. There is considerable diversity among these 
and many philosophically and morally contested areas. 
Here we will not enter into debates which cannot be 
settled by research evidence, but will simply note 
the main areas of contention. 

First, there are diverse views about the nature 
of knowledge and about how to access and use it. 

The power which people can exercise through thinking 
and communication also occupies many writers, who 
take positions ranging from various forms of elitism 
(intellectual, socio-cultural or spiritual) to an egalitarian 
concern forhuman rights. There are also distinct moral 
and ethical belief systems - with some writers taking a 
pragmatic, technological view aboutthe possible social 
and economic benefits of improved thinking; some 
espousing the values of a liberal-humanistic tradition; 
and others having a strong belief in rationalism. 

We have also commented on a spectrum of views about 
nature and nurture and about individual freedoms and 
state control. Finally, opinions differ widelyabout which 
aspects of thinking should be taughtand howthey 
should be taught. 



Of the contested issues listed above, those in the last 
paragraph are, to varying degrees, open to systematic 
enquiry and research. The first of these (the nature 
and nurture debate) has been researched in relation 
to adults more than the others. For example, Pederson, 
Plomin and McClearn (1994)found substantial and 
broadly similargenetic influences on both general 
and specific cognitive abilities. Several studies have 
shown that genetic influences on personality traits 
are somewhat lowerthan on cognitive abilities (Loehlin 
1992). What is not known about thinking and learning 
in adults is how fargenetic influences impose limits on 
achievement when motivation is high and good-quality 
personal and environmental support are provided. 

Most teachers accept that it is a core part of their role 
to take account of individual differences in learners, 
whether in terms of goals, preferences, ability, aptitude, 
orstyle of thinking and learning. However there is 
a paucity of research in the post-16 sectorto indicate 
whetherthere is much room for improvement in this 
area, which is whythe LSDA has also commissioned 
research reviews about learning styles (Coffield et al. 
2003a, 2003b). There is no doubt that teachers 
are attracted by the idea that better results maybe 
achievable if they capitalise on individual strengths, 
such as Gardner’s multiple intelligences. However in 
researching this area, there is a serious methodological 
difficulty in trying to control forthe catalytic effects 
of enthusiasm on the part of those who take up 
novel approaches. 

J ust as the evidence base on the best ways to meet 
the individual needs of post-16 learners is weak, 
little is known about how to optimise performance in 
teachers in ways which respect individual differences 
and build on personal strengths. Apart from differences 
in the nature and extent of teacherdirection and the 
facilitation of independent learning, teachers differ 
in otherways; for example, in creativity, in how they 
respond to a prescriptive curriculum, in their interest 
in abstract thinking, and in 'emotional intelligence'. 

There seems to be very little public debate about 
which aspects of thinking people believe to be 
important in adult life and how they can best be 
developed. Yet if the present decline in participation 
in adult education is to be reversed, the views and 
motivation of actual and potential participants need 
to be studied in depth. There is at present very little 
evidence on which to base policy in terms of the 
emphasis to be given to different kinds of thinking 
(eg procedural, critical, creative orcaring) in 
educational, vocational and recreational contexts. 



6.4 

What questions arise for further research? 

As we have seen, very little research into thinking 
skills in post-16 learners has been carried out in 
the UK. We therefore need to identify: (a) the contexts 
in which knowledge, understanding and productive 
thinking are most lacking; and (b) which current 
pedagogic approaches seem to be most effective 
in problematic areas. 

In Section 6.2, we identified a fundamental problem 
aboutthe contribution of theoryin education and 
training. The finding that theory-based interventions 
are typically no more effective than others makes 
it essential to evaluate theories and the ways in which 
they are used. In this respect, we believe that the 
present study has served a useful purpose, but it 
has not addressed the six issues raised towards the 
end of thatsection. This should be done by involving 
teachers in the development, application and evaluation 
of theory, ratherthan bya 'top down’ approach. 

We have not found any studies in which the 
communicability and practical utilityof a thinking 
skill framework in post-16 education and training has 
been formally assessed. Research on how readily 
teachers can learn to use thinking skill frameworks, 
on how they may need to modify them for classroom 
use and aboutthe impact of theiruse on teaching 
and learning should therefore be given high priority. 

We are notaware of any published UK studies in 
which syllabuses, course planning, assessmenttools 
or learning-related discourse have been analysed 
using a framework ortaxonomyof thinking skills. 

If an analysis of this kind led to significant changes 
in classroom practice, these could be evaluated, 
initially through pilot studies and then more widely. 

The positive message from international research 
findings aboutthe value of self-regulation and 
self-directed interventions reported in Section 6.1 
suggests that the communicability and utility for 
learners of the language of learning, thinking and 
instruction are even more important topics for research 
in post-16 education. Here, as in otherareas, there 
appears to be great potential for colla borati ve 
work between practitioners and research-trained 
professionals, first to agree appropriate content and 
then to find effective ways of communicating it, perhaps 
taking forward and extending the work of Mason (1999). 

In view of the importance of beliefs and motivation 
in thinking and learning, this is an area where a major 
research programme is required. 'One fits all' policies 
are likelyto be counterproductive if theydo not 
accommodate a range of approaches to teaching and 
learning, allowing for differences between contexts 
as well as for stylistic differences between teachers 
and learners. 
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There is clearly an urgent need to carryout further 
research into the effectiveness of pedagogical 
interventions intended to improve teacherand student 
thinking in the post-16 sector. Particular attention 
should be paid to interventions where there is 
already research evidence of generalisability. There is 
a pressing need to develop this work in contexts where 
increased social inclusion is one of the benefits sought, 
particularly in relation to contemporary initiatives in 
adult and community education (ACE) in relation to the 
development of adult numeracy and literacy, English 
forSpeakers of Other Languages (ESOL) and key skills. 

Avery promising area for continuing research and 
development is the introduction of thinking skills 
approaches into a range of courses in FE colleges, 
building on exemplars such as the work of Gregson 
(2003) and Duffy (2003). Should the early results prove 
encouraging, research will then be needed to studythe 
benefits of more radical reforms, such as applying the 
principles of a thinking community of enquiry throughout 
the education and training system: in policy-making; 
in curriculum design; in the education, training and 
support of teachers; in assessment; and in inspection. 

Thinking skills approaches are not necessarily 
labour-intensive, or limited to face-to-face forms 
of learning. Research is needed to establish what 
kinds of thinking and learning activity are best 
carried out in large groups and which in smallerones. 

At the same time, it is importantto find out (a) how 
distance-learning methods are best integrated with 
in-person class attendance; and (b) what difference 
information and communications technology (ICT) 
can make to the need for, and frequencyof, personal 
interaction in different-sized groups. 

Assessment is a powerful driver of educational change. 

If progress in thinking and learning and the benefits 
of thinking skills approaches are to be measured 
reliablyand with some validity, authentic, dynamic 
and ecologically relevant forms of assessment and 
evaluation must be devised. These must be suitable 
forthe assessment of group as well as individual 
activities and should be in electronic, paperand other 
forms. In the production and assessment of written 
work and argument, the value for learners and teachers 
of using the SOLO taxonomy (Biggs and Collis 1982; 
see Section 3.3.1) should be putto the test. 

In ourview, the first logical step is to work 
collaboratively with teachers in post-16 education, 
to explore the potential uses and further development 
of the new integrated framework presented in this 
report (Section 4.3). At the same time, teachers 
should be introduced to this and other frameworks 
and pedagogical approaches designed to develop 
thinking skills. It will then be possible to set up, through 
action research and large-scale studies, systematic 
evaluations of theirapplication and potential forwider 
use with post-16 learners across the UK. 



Ourconclusions need to be considered alongside 
those from the Universityof Strathclyde about post-16 
pedagogy and thinking skills (Livingston, Soden and 
Kirkwood 2003) and those from the Evidence forPolicy 
and Practice Information and Coordinating Centre 
(EPPI) systematic review (Higgins et al. 2003), in order 
to generate and develop new research. This should take 
several forms, including case studies, collaborative 
action research, randomised controlled trials and 
longitudinal process and outcome studies. 

6.5 

What are the implications of our researchfor 
policy-makers? 

In planning anychanges to the national system of 
post-16 education and training, the aims and objectives 
should be explicitly informed by the values that are 
widely associated with good thinking and meaningful 
learning. A balanced approach should be taken, 
so that critical, creative and caring thinking each play 
a part. Consideration should be given to developing new 
programmes focused on the study of human thinking, 
learning and behaviour. 

Courses and qualifications in which there is incomplete, 
unbalanced orfragmented coverage of relevant ways 
of developing (a) knowledge and (b) thinking should 
be modified. A theoretical framework of thinking and 
learning should inform anychanges made, perhaps 
beginning with key skills. 

Teacher-training courses should include a more solid 
grounding in theories of thinking and learning than they 
do at present. In view of the importance which many 
stakeholders give to the internalising of knowledge 
and skills bythe learner, theories which deal thoroughly 
with this issue should be included. 

In the initial training and post-experience 
development of teachers, theory should be integrated 
with experiential learning and professional practice 
in ways which strengthen critical, creative and 
caring dispositions. 'Communityof enquiry’ and 
'problem-based learning' are examples of models 
which appearto meetthese requirements and 
should therefore be fully evaluated in the context 
of teacher training. 

Dialogue and group discussion about how people 
think and learn should be a planned feature of college 
provision, available to all students at the start of their 
studies and on a regular basis. 

Because strategic thinking and reflection are of value 
in all kinds of learning and at all levels, they should 
form part of all education and training (including work 
in basic skills and programmes addressing the needs 
of all students who find learning difficult). 



Assessment should be designed to measure much 
more than the abilityto reproduce information and use 
it to demonstrate understanding. Productive, strategic 
and reflective thinking should also be assessed with 
some rigour and not only in learners preparing for 
higher-level qualifications. 

Continuing practitionerenquirythrough action 
research should be directed at finding ways of enabling 
people to build on previous learning and to make 
use of their skills and abilities in more areas of their 
lives. Research and evaluation should be seen as 
collaborative activities, involving all stakeholders, 
not least the learners themselves. 
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Glossary 



accommodation 

adapting actions to respond to new stimuli 
(in Piaget's theory) 

acculturated 

having taken on the ways of thinking and behaving 
typical of a social group 

affective 

characterised by emotion 
algorithm 

a precise rule (or set of rules) specifying how 
to solve a problem 

ampliative 

enlarging a conception by adding to what is already 
known 

analogical reasoning 

thinking that if things agree in some respects, 
they probably agree in others 

analytic 

focusing on the parts of a whole oron underlying 
basic principles 

assimilation 

absorbing new information and fitting it into existing 
knowledge (in Piaget’s theory) 

attributional training 

training which encourages the learner to attribute 
success orfailure to processes within theircontrol 
(such as effort rather than innate ability) 

behaviourist 

focusing only on observable stimuli and responses 
and the relations between them, discounting 
mental processes 

cognitive 

concerned with the psychological processes of 
perception, memory, thinking and learning 

conative 

expressive of effort and the will to achieve 
construct 

abstract orgeneral idea inferred from specific instances 
constructivist 

(educational sense) believing that knowledge or 
understanding is created when the learner integrates 
new information with prior knowledge; (philosophical 
sense) believing that reality does not exist outside 
ourconceptions, and that by conceptualising we 
'construct' it 

convergent thinking 

thinking directed at finding a single correct solution 
to a well-structured problem 

correlated 
mutually related 



correlation 

a measure indicating how fartwo variables are 
totally unconnected (zero correlation), orare negatively 
(eg -0.5) or positively related, as determined by 
underlying oroutside influences 

declarative knowledge 
knowledge of facts - knowing 'that' 

deductive 

reasoning from a general statement ordefinition to 
a particularinstance 

diagnosis 

identifying the nature orcausation of a problem 
dialectic 

involving a contradiction of ideas which acts as the 
determining factor in their interaction 

dichotomous 

dividing into two sharply distinguished parts or 
classifications 

disposition 

habit of mind or attitude 

divergentthinking 

exploratory thinking, seeking different possible ways 
of coping with ill-structured problems 

dynamic assessment 

finding out how far performance improves in response 
to teaching 

effect size 

a measure of difference or gain in average scores, 
whereby effect sizes of less than 0.2 (equivalent to a 
difference of 8 percentile points) are usually considered 
trivial, between 0.2 and 0.5 small, between 0.5 and 
0.8 moderate, and 0.8 or more large (equivalent to 
a difference of 28 percentile points) 

epistemology 

the philosophical study of theories of knowledge 
extrinsic motivation 

the desire to do something in order to obtain an 
external reward 

factor 

an underlying dimension orinfluence 
factor analysis 

a statistical technique which identifies underlying 
dimensions in a set of measures by finding groups of 
items which vary between individuals in similar ways 

far transfer 

successfully making use of knowledge or skills 
where the context and performance required are 
substantially different from those experienced in the 
learning situation 

field independence 

being able to see parts of a structure distinctly 
and objectively 





formative assessment 
evaluation carried out in the course of an activity 
in such a way that the information obtained is used 
to improve learning and/ or instruction 

g (general intelligence) 

a general cognitive ability factor which, in addition to 
specific abilities and skills, contributes to performance 
on a wide range of tasks 

genetic epistemology 

Piaget's theory about how children and young people 
move through a series of developmental stages, in 
which theirunderstanding of objects, relationships and 
concepts is limited by their powers of thought 

global 

not interested in detail: holistic 
heuristic 

rule-of-thumb strategy intended to increase the chances 
of solving a problem 

holistic 

perceiving a whole object orfocusing on the organic 
nature of a system 

inductive 

reasoning from particularfacts to a general conclusion 
intrinsic motivation 

the desire to do something forthe sake of the 
experience alone 

keyskilis 

government (England and Wales) guidelines on sixareas 
of skill deemed necessary for contemporary living - 
Communication, Application of Number, Information 
Technology, Working with Others, Problem Solving and 
Improving One’s Own Learning and Performance 

meta-analysis 

the process of synthesising a range of experimental 
results into a single estimate of effect size 

metacognition 

awareness and conscious use of the psychological 
processes involved in perception, memory, thinking 
and learning 

metaphysical 

dealing with highly abstract ideas about being and 
knowing which are not derived from the material world 

near transfer 

successfully applying knowledge or skills when the 
context and performance required are similarto those 
experienced in the learning situation 

pedagogy 

theoretical and procedural knowledge about teaching 
percentile 

a point on a scale below which a given percentage 
of a population will score 



phenomenological 

concerned with the recording and classification of 
subjective experience 

perception 

interpreting and understanding information received 
through the senses 

procedural knowledge 

knowledge of carrying out processes - knowing 'how' 
psychometric 

concerned with psychological measurement 
psychomotor 

relating to movements of the body associated with 
mental activity 

psychosocial 

relating to processes orfactors that are both social 
and psychological 

rationalist-constructivist tradition 

reliance on reason alone to construct one’s own 

understanding 

schema (plural: schemata) 

a structured mental representation consisting of related 
concepts 

self-efficacy 

having confidence in one’s abilityto succeed 
self-regulation 

the process of setting goals for oneself and then 
monitoring and evaluating progress 

Socratic questioning 

a deeply probing technique to uncover meaning, truth, 
understanding orbeliefs 

summative assessment 

evaluation of performance carried out at the end of 

a piece of work 

syllogistic reasoning 

a deductive inference consisting of two premises which 
are assumed to be true and a conclusion. Some women 
are mortal; some women are angelic; so some mortals 
are angelic is invalid 

taxonomy 

a principled classification of the elements of a domain 

transduction 

literal copying or mapping 

validity 

the quality of being well grounded in reality 
working memory 

the system which temporarily holds the information 
needed fora particulartask 
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Theoretical frameworks for thinking about thinking 



All-embracing family 

1950s 



1960s 



ED Educationalist 
PS Psychologist 
PH Philosopher 

A Deals with affective domain 
Cg Deals with cognitive domain 
Cn Deals with conation 
M Deals with metacognition 
E Aimed at educators 



1970s 



1980s Romiszowski’s analysis 
of knowledge and skills 
Romiszowski distinguishes between 
reproductive and productive learning 
in fourskill domains: cognitive, 
psychomotor, reactive and 
interactive. He identifies 12 abilities 
which maybe used in perception, 
recall, planning and performance. 
ACgCn M E/ ED 



1990s Hauenstein’s conceptual 


J onassen and Tessmer's 


Vermunt and Verloop's 


framework for educational 


taxonomy of learning outcomes 


categorisation of learning 


objectives 


The major categories in this 


activities 


Acquisition, assimilation, 


taxonomyare: declarative 


The cognitive categories are: 


adaptation, performance and 


knowledge, structural knowledge, 


relating/ structuring; analysing; 


aspiration are successive levels of 


cognitive component skills, 


concretising/ applying; 


learning in the cognitive, affective 


situated problem solving, 


memorising/ rehearsing; critical 


and psychomotor domains. At 


knowledge complexes, ampliative 


processing; selecting. The affective 


each level and in each domain, 


skills, self-knowledge, reflective 


categories cover motivation and 


Hauenstein identifies processes 


self-knowledge, executive control, 


the management of feelings. 


which help to build understanding, 


motivation (disposition) and 


The regulative categories are an 


skills and dispositions. 


attitude. 


elaboration of ‘plan-do-review'. 


ACg Cn E / ED 


ACg Cn M E/ PS 


A Cg Cn M E/ PS 



2000s 



Demetriou's model of mind, 
personality and self 
For Demetriou, mind and personality 
interact at all levels of self-oriented 
and environment-oriented systems. 
There are long-term and working 
self systems, representational and 
regulatory. Progress through Piaget's 
stages can be seen in categorical, 
quantitative, causal, spatial, verbal, 
social and drawing ‘modules'. 

A Cg M / PS 



M arzano's new taxonomy of 
educational objectives 
The self system examines the 
importance of new knowledge, 
ability to learn and emotions linked 
with knowledge and motivation. 

The metacognitive system specifies 
learning goals and monitors 
execution, clarity and accuracy. 

The cognitive system deals with 
retrieval, comprehension, analysis 
and knowledge utilisation. 

A Cg Cn M E/ ED 



Sternberg's model of abilities 
as developing expertise 
This model includes the 
analytical, creative and 
practical aspects of ‘successful 
intelligence’: metacognition, 
learning skills, knowledge, 
motivation and the influence 
of context. 

ACgCn M E/ PS 





Instructional design family 



1950s 



Bloom's taxonomy 
of educational objectives 
(cognitive domain) 

This framework is a wayof 
classifying educational goals 
in terms of complexity. The 
intellectual abilities and skills 
of comprehension, application, 
analysis, synthesis and 
evaluation are applied to 
and help build knowledge. 

Cg E/ PS 



Feuerstein’s theory 
of mediated learning through 
instrumental enrichment 
Building on his belief in 
cognitive modifiability, 

Feuerstein developed the 
concept of a mediated learning 
experience in which the mediator 
uses prescribed tasks to 
promote thinking rather than 
rote learning. 

ACg M E/ PS 



1960s Ausubel and Robinson's 
six hierarchically ordered 
categories 

These are: representational 
learning, concept learning, 
propositional learning, 
application, problem solving 
and creativity. 

Cg E / PS 



B arrett’s taxonomy 
of cognitive and affective 
dimensions of reading 
comprehension 
The main categories used 
are: literal comprehension, 
reorganisation, inferential 
comprehension, evaluation 
and appreciation. 

A Cg E / ED 



Gagne's eight types of 
learning and five types 
of learned capability 
Gagne set out an eight-level 
hierarchy of learning types, 
with problem solving at the top. 
He also identified five domains 
of learning: motor skills, 
verbal information, intellectual 
skills, cognitive strategies 
and attitudes. 

ACg M E/ PS 



Gerlach and Sullivan's 
taxonomy of commonly 
taught behaviours 
This is a framework with 
six categories of student 
behaviour: name, order, 
identify (classify), describe, 
demonstrate and construct, 
Cg E/ PS 



1970s 



De Block's three-dimensional 
framework 

This framework is focused 
on learning Latin. The first 
dimension has four categories: 
knowledge, comprehension, 
application and integration. 

The second dimension is 
content-based, and the third 
deals with the development 
of personal qualities, 

A Cg Cn E / ED 



Flannah and M ichaelis's 
comprehensive framework 
for instructional objectives 
The cognitive, psychomotor 
and affective domains 
are covered. Interpreting, 
comparing, classifying, 
generalising, inferring, 
analysing, synthesising, 
hypothesising, predicting 
and evaluating are listed 
as intellectual processes. 
ACg E/ ED 



F Williams' model for 
developing thinking and 
feeling processes 
This three-dimensional 
cross-curricular model seeks 
to encourage creativity. Teachers 
can use 18 teaching modes 
to promote fluency, flexibility, 
originality, elaboration, curiosity, 
risk taking, complexity and 
imagination. 

A Cg Cn E / ED 



R Williams' behavioural 
typology of educational 
objectives (cognitive domain) 
Williams lists types of content 
(facts, concepts, principles 
and procedures) and ways of 
testing the learning involved in: 
memorisation, summarisation, 
instantiation, prediction, 
application and evaluation. 

Cg E/ ED 



1980s Beyer's taxonomy 


Biggs and Collis's SOLO 


Bruce's modification of 


M arzano's dimensions 


This is a three-level 


taxonomy 


Bloom's taxonomy 


of thinking 


framework in which eight 


This is an assessment tool 


Instead of having a separate 


This framework has five 


fundamental skills (based on 


looking at the structure of the 


knowledge category, Bruce 


interrelated dimensions: 


Bloom) are used in at leastten 


observed learning outcome. 


inserts the sub-categories of 


metacognition, critical and 


critical-thinking operations 


Pre-structural responses 


knowledge into Bloom's original 


creative thinking, thinking 


in orderto conceptualise, make 


betray limited understanding 


categories of application, 


processes, core thinking 


decisions, and solve problems. 


compared with uni-structural 


analysis, synthesis and 


skills, and the relationship 


Metacognition (thinking about 


and multi-structural responses. 


evaluation. 


of content-area knowledge 


thinking) is emphasised. 


Relational and extended 


Cg E/ ED 


to thinking. 


Cg Cn / PS 


abstract responses are 
qualitatively superior. 
Cg E/ ED 




ACg Cn M E / ED 



1990s 



Presseisen’s models of basic, 
complex and metacognitive 
thinking skills 
Presseisen lists five 'basic 
processes' which are used 
in problem solving, decision 
making, critical thinking 
and creative thinking. She also 
lists six metacognitive thinking 
skills involved in strategy 
selection, understanding 
and monitoring. 

Cg M E/ ED 



Wiggins and M cTighe's six 
facets of understanding 
These authors list three 
performance facets (explain, 
interpret and apply) and 
three qualitative indicators 
of understanding (perspective, 
empathyand self-knowledge). 
ACg M E/ ED 



Wong and Williams' 
integrated thinking model 
Knowledge is used in basic 
thinking, reorganised in 
critical thinking and generated 
in creative thinking. Critical 
thinking consists of three 
sub-categories: analysing, 
connecting and evaluating. 

Cg M E / ED 



2000s 



Anderson and Krathwohl’s 
revision of Bloom's taxonomy 
Bloom's taxonomy has been 
refined and developed into 
a two-dimensional framework 
using six cognitive processes 
and four knowledge categories. 
There is an emphasis on aligning 
learning objectives with learning 
activities and assessment. 

Cg M E / PS 



Gouge and Yates ' A RTS 
proj ect taxonomies of arts 
reasoning and thinking skills 
A matrix of Piaget’s levels 
(concrete, concrete transitional 
and formal operational thinking) 
and reasoning skills is used 
to create educational objectives 
forthe visual arts, music 
and drama. 

Cg M E/ ED 



Passig’s taxonomy of IT- 
mediated future thinking 
skills 

This is an extension of 
Bloom's taxonomy, with the 
additional category melioration - 
combining, adapting and using 
information to solve problems 
in accordance with cultural and 
personal meanings. 

Cg E / ED 
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ED Educationalist 
PS Psychologist 
PH Philosopher 

A Deals with affective domain 
Cg Deals with cognitive domain 
Cn Deals with conation 
M Deals with metacognition 
E Aimed at educators 



I nstructional design family 
1950s 



1960s 



1970s 



M errill's component 
displaytheory 

Merrill classifies learning along 
two dimensions: performance 
(remember instance, remember 
generality, use, find) and content 
(facts, concepts, procedures, 
and principles). 

Cg E / PS 



Quellmalz's framework 
of thinking skills 
This framework lists 
five cognitive processes 
(recall, analysis, comparison, 
inference/ interpretation 
and evaluation) and three 
metacognitive processes 
(planning, monitoring 
and reviewing/ revising). 

Cg M E / ED 



Stahl and M urphy's domain 
of cognition 
These authors set out a 
multi-stage model of information 
processing from preparation 
to generation. Theyalso 
identify 2 1 cognitive processes 
(eg classifying, organising, 
selecting, utilising, verifying) 
which maybe used singlyorin 
combinations at different levels. 
ACg Cn E/ PS 



Williams and Haladyna's 
typology for higher-level 
test items 
This framework is 
three-dimensional, with 
content, task, and response 
mode dimensions. The task 
categories are: reiteration, 
summarisation, illustration, 
prediction, application and 
evaluation (cf Williams 1977). 
Cg E/ PS 



1980s 



1990s 



2000s 




Critical/productive thinking family 
1950s Altshuller'sTRIZtheory 

of inventive problem solving 
There are four main steps: 
problem definition; 
problem-solving tool selection; 
generating solutions; 
evaluating solutions. 

A specific problem is an instance 
of a generic problem which 
is solved when the appropriate 
generic solution is returned 
to a specific solution. 

Cg M E/ ED 



Kepner-Tregoe tenets of 
effective decision making - 
rational thinking 
Rational processes and practical 
steps are listed in simple 
language underthe following 
headings: situation appraisal, 
problem analysis, decision 
analysis and potential problem 
(opportunity) analysis. 

Cg E / PS 



1970s Lipman's three modes 

of thinking and four main 
varieties of cognitive skill 
Judgementand reasoning can 
be strengthened through critical, 
creative and caring thinking. 

In education, the four major 
varieties of higher-order thinking 
relate to: enquiry, reasoning, 
information organising and 
translation. 

A Cg M E/ PH 



1960s Allen, Feezel and Kauffie's 
taxonomy of concepts and 
critical abilities related to the 
evaluation of verbal 
arguments 

Twelve abilities are involved 
in the recognition, analysis 
and evaluation of arguments. 
Truth claims depend on 
testimony and reasons. People 
should not be misled by rhetoric 
orthe misuse of language. 



1980s 



Baron's model of the 
good thinker 

The most important components 
of the model are the three 
conscious search processes - 
for goals, for possibilities and 
for evidence. Good thinking 
and the dispositions underlying 
it are to some extent teachable. 
ACgM E/ PS 



Ennis's taxonomy of critical 
thinking dispositions and 
abilities 

'Critical thinking is reasonable 
reflective thinking that is 
focused on what to believe 
or do.' For Ennis, the basic areas 
of critical thinking are clarity, 
basis, inference and interaction. 
He lists 12 relevant dispositions 
and 15 abilities. 

Cg M E / PH 



Gubbins' matrix 
of thinking skills 
This is a composite list of 'core' 
critical thinking skills, based on 
other published lists. The skills 
are grouped underthe following 
headings: problem solving, 
decision making, inferences, 
divergent thinking, evaluating, 
philosophy and reasoning. 

Cg M E / ED 



Halpern's reviews 
of critical thinking skills 
and dispositions 
Halpern's skill categories are: 
memory, thought and language, 
deductive reasoning, argument 
analysis, hypothesis testing, 
likelihood and uncertainty, 
decision making, problem 
solving, and creative thinking. 
She also lists six relevant 
dispositions. 

Cg Cn M E / PS 



1990s 


J ewell's reasoning taxonomy 


Perkins, Jay and Tishman's 




for gifted children 


dispositional theory 




Jewell's taxonomy has three 


of thinking 




fields: objectives of reasoning, 


Seven dispositions are listed: 




reasoning strategies and 


to be broad and adventurous; 




reasoning dispositions. The 


to sustain intellectual 




disposition to adopt thinking 


curiosity; to clarifyand 




about thinking (metacognition) 


seek understanding: 




as a habit is very important. 


to be 'planful' and strategic; 




A Cg M E/ PH 


to be intellectually careful; 
to seek and evaluate reasons; 
to be metacognitive. 

ACgCn M E/ PS 



2000s 
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ED Educationalist 
PS Psychologist 
PH Philosopher 

A Deals with affective domain 
Cg Deals with cognitive domain 
Cn Deals with conation 
M Deals with metacognition 
E Aimed at educators 



Critical/productive thinking family 
1950s 



1960s 



1970s 



Paul's model of 1980s 

critical thinking 

The model has four parts: 

elements of reasoning, 

standards of critical thinking, 

intellectual abilities and 

intellectual traits. The first 

three parts focus on what 

is essential to critical thinking 

and the fourth on what it is 

to be a critical thinker. 

ACgCn M E/ PH 



1990s 



2000s 




